Discovery of connected devices to determine control capabilities and meta-information

ABSTRACT

The present disclosure relates to techniques for interacting with an electronic device. More specifically, the present disclosure relates to initiating action(s) using an interface configured to control operation of IoT devices. An example embodiment includes receiving, at a computing device, a communication including a unique identifier for a network device connected to a network, using the unique identifier to determine a resource bundle for the network device, the resource bundle defining an interface module for the network device, wherein the interface module is configured to display one or more interface elements usable to control the network device, and transmitting the resource bundle, wherein when the resource bundle is received, the resource bundle facilitates generating the interface module.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/362,462, filed on Nov. 28, 2016, which is a continuation-in-part ofU.S. application Ser. No. 14/959,192, filed on Dec. 4, 2015, whichclaims the benefit of and priority to U.S. Provisional Application No.62/087,458, filed on Dec. 4, 2014, and U.S. Provisional Application No.62/088,460, filed on Dec. 5, 2014.

This application is a continuation of U.S. application Ser. No.15/362,462, filed on Nov. 28, 2016, which is also a continuation-in-partof U.S. application Ser. No. 15/208,162 filed on Jul. 12, 2016, which isa continuation of U.S. application Ser. No. 14/525,443, filed on Oct.28, 2014, now issued as U.S. Pat. No. 9,426,118.

This application is a continuation of U.S. application Ser. No.15/362,462, filed on Nov. 28, 2016, which is also a continuation-in-partof U.S. application Ser. No. 15/015,887, filed on Feb. 4, 2016, whichclaims the benefit of and priority to U.S. Provisional Application No.62/111,827, filed on Feb. 4, 2015.

This application is a continuation of U.S. application Ser. No.15/362,462, filed on Nov. 28, 2016, which is also a continuation-in-partof U.S. application Ser. No. 14/959,380, filed on Dec. 4, 2015, whichclaims the benefit of and priority to U.S. Provisional Application No.62/087,466, filed on Dec. 4, 2014, U.S. Provisional Application No.62/087,753, filed on Dec. 4, 2014, U.S. Provisional Application No.62/087,756, filed on Dec. 4, 2014, U.S. Provisional Application No.62/088,297, filed on Dec. 5, 2014, and U.S. Provisional Application No.62/115,467, filed on Feb. 12, 2015.

This application is a continuation of U.S. application Ser. No.15/362,462, filed on Nov. 28, 2016, which is also a continuation-in-partof U.S. application Ser. No. 14/326,393 filed on Jul. 8, 2014, which isa continuation of U.S. application Ser. No. 14/267,834, filed on May 1,2014.

This application is a continuation of U.S. application Ser. No.15/362,462, filed on Nov. 28, 2016, which is also a continuation-in-partof U.S. application Ser. No. 14/292,066 filed on May 30, 2014.

This application is a continuation of U.S. application Ser. No.15/362,462, filed on Nov. 28, 2016, which is also a continuation-in-partof U.S. application Ser. No. 14/286,439 filed on May 23, 2014, which isa continuation of U.S. application Ser. No. 14/254,689 filed on Apr. 16,2014.

All of these applications are hereby incorporated by reference in theirentireties for all purposes.

FIELD

The present disclosure relates to techniques for interacting with anelectronic device. More specifically, the present disclosure relates toinitiating action(s) using an interface configured to control operationof IoT devices.

BRIEF SUMMARY

Embodiments of the present invention may include a computer-implementedmethod, comprising receiving, at a computing device, a communicationincluding a unique identifier for a network device connected to anetwork, using the unique identifier to determine a resource bundle forthe network device, the resource bundle defining an interface module forthe network device, wherein the interface module is configured todisplay one or more interface elements usable to control the networkdevice, and transmitting the resource bundle, wherein when the resourcebundle is received, the resource bundle facilitates generating theinterface module.

In another aspect, when the resource bundle is received, the resourcebundle facilitates executing an application that generates the interfacemodule. In another aspect, when the resource bundle is received by anaccess device, the resource bundle facilitates executing an applicationthat generates the interface module. In another aspect, the methodfurther comprises receiving a request for the resource bundle from anaccess device, and transmitting the resource bundle to the requestingaccess device, wherein when the resource bundle is received by theaccess device, the resource bundle facilitates executing an applicationat the access device that generates the interface module. In anotheraspect, the method further comprises receiving an updated resourcebundle for the network device, registering the updated resource bundle,and transmitting the updated resource bundle, wherein when the resourcebundle is received, the resource bundle facilitates generating anupdated interface module for the network device. In another aspect, whenthe updated resource bundle is received at an access device, the accessdevice generates the updated interface module for the network device. Inanother aspect, the one or more interface elements correspond tocontrols or subcontrols of the network device. In another aspect, theresource bundle defines the interface module to include a contextualmenu for the network device, the contextual menu corresponding to asecondary control of the network device, and wherein the secondarycontrol is associated with a primary control of the network device. Inanother aspect, interface elements are selectable, wherein selecting aninterface element displays a menu, and wherein the menu includes achangeable setting associated with the network device. In anotheraspect, interface elements display a state of the network device.

Alternative embodiments of the present invention may include one or moredata processors, and a non-transitory computer-readable storage mediumcontaining instructions which when executed on the one or more dataprocessors, cause the one or more data processors to perform operations.The operations may include receiving a communication including a uniqueidentifier for a network device connected to a network, using the uniqueidentifier to determine a resource bundle for the network device, theresource bundle defining an interface module for the network device,wherein the interface module is configured to display one or moreinterface elements usable to control the network device, andtransmitting the resource bundle, wherein when the resource bundle isreceived, the resource bundle facilitates generating the interfacemodule.

Alternative embodiments of the present invention may include acomputer-program product tangibly embodied in a non-transitorymachine-readable storage medium of a computing device, includinginstructions configured to cause one or more data processors to receivea communication including a unique identifier for a network deviceconnected to a network, use the unique identifier to determine aresource bundle for the network device, the resource bundle defining aninterface module for the network device, wherein the interface module isconfigured to display one or more interface elements usable to controlthe network device, and transmit the resource bundle, wherein when theresource bundle is received, the resource bundle facilitates generatingthe interface module.

Embodiments of the present invention may include a computer-implementedmethod, comprising receiving, by a computing device, informationassociated with a device connected to a network, wherein the informationincludes criteria for operation of the device, storing the informationon a cache of the computing device, receiving updated informationassociated with the device, wherein the updated information includesupdates to the stored information, modifying the information stored onthe cache using the updated information, detecting a request for currentinformation associated with the device, retrieving the currentinformation from the information stored on the cache, and transmittingthe retrieved current information to the network.

In another aspect, the information includes device informationidentifying devices connected to the network. In another aspect, therequest includes a discovery request for current device informationidentifying devices connected to the network. In another aspect, thecomputing device is included in a router device. In another aspect, thecriteria includes operation criteria and instruction criteria. Inanother aspect, the method may further comprise transmitting, to thenetwork, a request for updates to the stored information, wherein therequest indicates a version of the information currently stored on thecache. In another aspect, the updated information is received from adevice connected to the network. In another aspect, modifying theinformation stored on the cache includes identifying a differencebetween the updated information and the information stored on the cache,and updating the cache with the updated information based on theidentified difference. In another aspect, requests for currentinformation associated with a device are received and transmitted usinga cloud network. In another aspect, the method may further comprisedetermining whether an amount of computing resources available for thecomputing device satisfies a resource threshold, and determining whetherupdates exists for the information stored on the cache when the resourcethreshold is satisfied.

Alternative embodiments of the present invention may include a system,comprising one or more data processors, and a non-transitorycomputer-readable storage medium containing instructions which whenexecuted on the one or more data processors, cause the one or moreprocessors to perform operations. The operations may include receivinginformation associated with a device connected to a network, wherein theinformation includes criteria for operation of the device, storing theinformation on a cache of the computing device, receiving updatedinformation associated with the device, wherein the updated informationincludes updates to the stored information, modifying the informationstored on the cache using the updated information, detecting a requestfor current information associated with the device, retrieving thecurrent information from the information stored on the cache, andtransmitting the retrieved current information to the network.

Alternative embodiments of the present invention may include acomputer-program product tangibly embodied in a non-transitorymachine-readable storage medium, including instructions configured tocause a data processing apparatus to receive, by a computing device,information associated with a device connected to a network, wherein theinformation includes criteria for operation of the devices, store theinformation on a cache of the computing device, receive updatedinformation associated with the device, wherein the updated informationincludes updates to the stored information, modify the informationstored on the cache using the updated information, detect a request forcurrent information associated with the device, retrieve the currentinformation from the information stored on the cache, and transmit theretrieved current information to the network.

Embodiments of the present invention may include a computer-implementedmethod, comprising determining, by a computing device, that a device isconnected to a network, generating a graphical interface, wherein thegraphical interface includes an interactive area corresponding to thedevice, displaying the graphical interface, receiving inputcorresponding to an interaction with the interactive area, wherein theinput is associated with an adjustment for controlling operation of thedevice, detecting continued interaction with the interactive area,wherein the continued interaction causes an action to be performed,performing the action, and transmitting the adjustment, whereinoperation of the device is controlled using the transmitted adjustment.

In another aspect, the input corresponds to a pre-determined timeperiod, and wherein the continued interaction exceeds the pre-determinedtime period. In another aspect, actions are related to adjustments. Inanother aspect, adjustments and actions are device specific. In anotheraspect, the interaction and the continued interaction are performedusing one or more contacts with the interactive area. In another aspect,continuous contact with the interactive area is maintained during theinteraction and the continued interaction. In another aspect, the actionincludes sending a notification related to the operation of the device.In another aspect, actions include controlling operation of one or moredevices. In another aspect, wherein the method may further comprisereceiving criteria for controlling operation of devices in the network,and using the criteria to determine one or more actions to be performed.In another aspect, the criteria corresponds to a mode for controllingoperation of devices connected to the network, and wherein modesincludes home modes, away modes, or sleep modes. In another aspect,performing the action may include displaying an additional graphicalinterface including an interactive area for receiving criteria usable tomodify an action. In another aspect, the action is a scheduled action.In another aspect, performing the action may include transmitting arequest including an adjustment for controlling operation of devicesconnected to the network. In another aspect, the method may furthercomprise receiving a response including indications of devices that havebeen adjusted, and modifying the graphical interface based on theresponse.

Alternative embodiments of the present invention may include a system,comprising one or more data processors, and a non-transitorycomputer-readable storage medium containing instructions, which whenexecuted on the one or more data processors, cause the one or moreprocessors to perform operations. The operations may include determiningthat a device is connected to a network, generating a graphicalinterface, wherein the graphical interface includes an interactive areacorresponding to the device, displaying the graphical interface,receiving input corresponding to an interaction with the interactivearea, wherein the input is associated with an adjustment for controllingoperation of the device, detecting continued interaction with theinteractive area, wherein the continued interaction causes an action tobe performed, performing the action, and transmitting the adjustment,wherein operation of the device is controlled using the transmittedadjustment.

Alternative embodiments of the present invention may include acomputer-program product tangibly embodied in a non-transitorymachine-readable storage medium. The machine-readable storage mediumincluding instructions configured to cause a data processing apparatusto determine that a device is connected to a network, generate agraphical interface, wherein the graphical interface includes aninteractive area corresponding to the device, display the graphicalinterface, receive input corresponding to an interaction with theinteractive area, wherein the input is associated with an adjustment forcontrolling operation of the device, detect continued interaction withthe interactive area, wherein the continued interaction causes an actionto be performed, perform the action, and transmit the adjustment,wherein operation of the device is controlled using the transmittedadjustment.

Embodiments of the present invention may include a computer-implementedmethod, comprising receiving, by a computing device, input correspondingto an interaction with an interactive area on an interface, wherein theinterface corresponds to a device, and wherein the input corresponds toan action for controlling operation of the device, using the input todetermine the action for controlling operation of the device,transmitting data indicating the action for controlling operation of thedevice, wherein transmitting the data causes the action to be performed,determining that the interaction with the interactive area exceeds athreshold time period, and performing the action when the interactionexceeds the threshold time period.

In another aspect, actions are related to adjustments. In anotheraspect, actions are device specific. In another aspect, the interactionis performed using one or more contacts with the interactive area. Inanother aspect, actions include sending notifications related tooperation of devices. In another aspect, actions include controllingoperation of devices. In another aspect, the method may further comprisereceiving criteria for controlling operation of devices, and using thecriteria to determine actions to be performed. In another aspect, thecriteria correspond to a mode for controlling operation of devicesconnected to the network, and wherein modes include home modes, awaymodes, or sleep modes. In another aspect, the method may furthercomprise receiving criteria usable to modify actions. In another aspect,actions are performed according to a schedule. In another aspect, theaction causes operation of devices to be controlled, and whereinperforming action includes transmitting data indicating the action tocontrol operation of devices. In another aspect, the method may furthercomprise transmitting a status including indications of devices thathave been controlled.

Alternative embodiments of the present invention may include a systemcomprising one or more data processors, and a non-transitorycomputer-readable storage medium containing instructions, which whenexecuted on the one or more data processors, cause the one or moreprocessors to perform operations. The operations may include receivinginput corresponding to an interaction with an interactive area on aninterface, wherein the interface corresponds to a device, and whereinthe input corresponds to an action for controlling operation of thedevice, using the input to determine the action for controllingoperation of the device, transmitting data indicating the action forcontrolling operation of the device, wherein transmitting the datacauses the action to be performed, determining that the interaction withthe interactive area exceeds a threshold time period, and performing theaction when the interaction exceeds the threshold time period.

Alternative embodiments of the present invention may include acomputer-program product tangibly embodied in a non-transitorymachine-readable storage medium, including instructions configured tocause a data processing apparatus to receive input corresponding to aninteraction with an interactive area on a interface, wherein theinterface corresponds to a device, and wherein the input corresponds toan action for controlling operation of the device, use the input todetermine the action for controlling operation of the device, transmitdata indicating the action for controlling operation of the device,wherein transmitting the data causes the action to be performed,determine that the interaction with the interactive area exceeds athreshold time period, and perform the action when the interactionexceeds the threshold time period.

Embodiments of the present invention may include a computer-implementedmethod, comprising determining, at a network device, that the networkdevice is connected to a network, receiving data determined at an accessdevice, wherein the data indicates an adjustment of an attribute relatedto operation of the network device, and wherein the data indicates athreshold time period associated with the adjustment, detecting inputcorresponding to an interaction with an interactive area of an interfacecorresponding to the network device, determining, using the detectedinput, that the interaction with the interactive area of the interfaceexceeds the threshold time period, and sending information indicatingthe adjustment when the interaction exceeds the threshold time period,wherein the sent information causes operation of the network device tobe controlled using the adjustment.

In another aspect, the data is determined based on input correspondingto an interaction with an interactive area of a graphical interfacedisplayed by the access device. In another aspect, the interaction isperformed by one or more contacts with the interactive area. In anotheraspect, the interaction with the interactive area is continuous andoccurs for a duration of the threshold time period. In another aspect,the method may further comprise determining a status related tooperation of the network device using the adjustment, and sending amessage indicating that the status, wherein the sent message causes amodification to a graphical interface displayed by the access device,and wherein the modification includes displaying the status. In anotheraspect, the method may further comprise generating a graphicalinterface, wherein the graphical interface includes an interactive areato configure operation of one or more network devices connected to thenetwork, displaying the graphical interface, and determining additionaldata based on input corresponding to an interaction with the interactivearea included in the graphical interface, wherein the additional dataindicates a modification to the threshold time period associated withthe adjustment. In another aspect, the method may further comprisedetermining a status related to operation of the network device usingthe adjustment, and modifying the graphical interface to indicate thatthe determined status. In another aspect, adjustments are devicespecific. In another aspect, the method may further comprise sending theinformation indicating the adjustment includes transmitting theinformation.

Alternative embodiments of the present invention may include a systemcomprising one or more data processors, and a non-transitorycomputer-readable storage medium containing instructions, which whenexecuted on the one or more data processors, cause the one or moreprocessors to perform operations. The operations may include determiningthat a network device is connected to a network, receiving datadetermined at an access device, wherein the data indicates an adjustmentof an attribute related to operation of the network device, and whereinthe data indicates a threshold time period associated with theadjustment, detecting input corresponding to an interaction with aninteractive area of an interface corresponding to the network device,determining, using the detected input, that the interaction with theinteractive area of the interface exceeds the threshold time period, andsending information indicating the adjustment when the interactionexceeds the threshold time period, wherein the sent information causesoperation of the network device to be controlled using the adjustment.

Alternative embodiments of the present invention may include acomputer-program product tangibly embodied in a non-transitorymachine-readable storage medium, including instructions configured tocause a data processing apparatus to determine that a network device isconnected to a network, receive data determined at an access device,wherein the data indicates an adjustment of an attribute related tooperation of the network device, and wherein the data indicates athreshold time period associated with the adjustment, detect inputcorresponding to an interaction with an interactive area of an interfacecorresponding to the network device, determine, using the detectedinput, that the interaction with the interactive area of the interfaceexceeds the threshold time period, and send information indicating theadjustment when the interaction exceeds the threshold time period,wherein the sent information causes operation of the network device tobe controlled using the adjustment.

Embodiments of the present invention may include a gateway device,comprising one or more processors, and a non-transitory machine-readablestorage medium containing instructions which when executed on the one ormore data processors, cause the one or more processors to performoperations. The operations may include detecting a network deviceconnected to the gateway device, wherein the gateway device providescommunication routing for the network device, and probing the networkdevice for meta-information and control capabilities of the networkdevice, and a receiver configured to receive the meta-information andthe control capabilities of the network device, wherein themeta-information includes identifying information of the network device,wherein the non-transitory machine-readable storage medium furthercontains instructions which when executed on the one or more dataprocessors, cause the one or more processors to perform operationsincluding, determining a device specific interface object for thenetwork device using the meta-information, determining one or moreprimary control capabilities corresponding to the network device, andtransmitting a device specific interface object module and the one ormore primary control capabilities, wherein the device specific interfaceobject module and the one or more primary control capabilitiesfacilitate display of a gateway device interface, wherein the gatewaydevice interface displays the device specific interface object and acontrol object, wherein the control object allows control of the networkdevice using the one or more primary control capabilities, and whereinthe one or more primary control capabilities include less than allcontrol capabilities of the network device that are provided in aproprietary interface of the network device.

In another aspect, the meta-information includes the device specificinterface object module. In another aspect, the meta-informationincludes a unique identifier for the network device, and furthercomprising instructions which when executed on the one or more dataprocessors, cause the one or more processors to perform operations. Theoperations may include transmitting the unique identifier to a cloudserver, receiving information identifying a type of the network device,and determining the device specific interface object for the networkdevice using the information identifying the type of the network device.In another aspect, determining the device specific interface object forthe network device further comprises monitoring behavior characteristicsof the network device, determining a type of the network device based onthe monitored behavior characteristics, and determining the devicespecific interface object for the network device using the determinedtype of the network device. In another aspect, instructions which whenexecuted on the one or more data processors, cause the one or moreprocessors to perform operations including transmitting themeta-information of the network device to a second network device. Inanother aspect, the device specific interface object includes thecontrol object, and wherein the device specific interface object isselectable to control the network device using the one or more primarycontrol capabilities. In another aspect, the gateway device interfaceincludes a proprietary interface object, wherein the proprietaryinterface object is selectable to launch the proprietary interface ofthe network device to allow execution of a control capability outside ofthe primary control capabilities. In another aspect, the gateway deviceinterface displays information corresponding to a plurality of networkdevices connected to the gateway device.

Alternative embodiments of the present invention may include acomputer-implemented method, comprising, detecting, by a gateway device,a network device connected to the gateway device, wherein the gatewaydevice provides communication routing for the network device, probingthe network device for meta-information and control capabilities of thenetwork device, receiving the meta-information and the controlcapabilities of the network device, wherein the meta-informationincludes identifying information of the network device, determining adevice specific interface object for the network device using themeta-information, determining one or more primary control capabilitiescorresponding to the network device, and transmitting a device specificinterface object module and the one or more primary controlcapabilities, wherein the device specific interface object module andthe one or more primary control capabilities facilitate display of agateway device interface, wherein the gateway device interface displaysthe device specific interface object and a control object, wherein thecontrol object allows control of the network device using the one ormore primary control capabilities, and wherein the one or more primarycontrol capabilities include less than all control capabilities of thenetwork device that are provided in a proprietary interface of thenetwork device.

Alternative embodiments of the present invention may include acomputer-program product tangibly embodied in a non-transitorymachine-readable storage medium of a gateway device, includinginstructions configured to cause one or more data processors to detect anetwork device connected to the gateway device, wherein the gatewaydevice provides communication routing for the network device, probe thenetwork device for meta-information and control capabilities of thenetwork device, receive the meta-information and the controlcapabilities of the network device, wherein the meta-informationincludes identifying information of the network device, determine adevice specific interface object for the network device using themeta-information, determine one or more primary control capabilitiescorresponding to the network device, and transmit a device specificinterface object module and the one or more primary controlcapabilities, wherein the device specific interface object module andthe one or more primary control capabilities facilitate display of agateway device interface, wherein the gateway device interface displaysthe device specific interface object and a control object, wherein thecontrol object allows control of the network device using the one ormore primary control capabilities, and wherein the one or more primarycontrol capabilities include less than all control capabilities of thenetwork device that are provided in a proprietary interface of thenetwork device.

Embodiments of the present invention may include a computer-implementedmethod, comprising, determining, by a computing device, a current statusassociated with a device on a network, wherein the current statusindicates availability of the device on the network, generating agraphical interface, wherein the graphical interface includes aninteractive element corresponding to the device, and wherein theinteractive element indicates the current status associated with thedevice, displaying the graphical interface, determining a change to thecurrent status, wherein determining the change includes determining atype of status corresponding to the change, determining an updatedstatus associated with the device, determining that the device isassociated with the updated status for a threshold time period, andmodifying the graphical interface according to the type of status whenthe threshold time period is reached, wherein the modified graphicalinterface includes a modified interactive element corresponding to thedevice, and wherein the modified interactive element indicates theupdated status associated with the device.

In another aspect, the modified interactive element has a modifiedgraphical appearance that indicates the updated status. In anotheraspect, a threshold time period corresponds to a type of status. Inanother aspect, the method may further comprise determining that thetype of status corresponds to a firmware update on the device, whereinthe modified interactive element has a modified graphical appearancethat indicates the firmware update. In another aspect, interaction withan interactive element in the graphical interface controls operation ofa device corresponding to the interactive element. In another aspect,when the updated status corresponds to unavailability of the device,modifying the graphical interface includes disabling interaction withthe modified interactive element. In another aspect, modifying thegraphical interface includes removing the interactive element from thegraphical interface when the updated status corresponds tounavailability of the device. In another aspect, the method may furthercomprise upon detecting that that device is available on the network,modifying the graphical interface to include the interactive elementthat was previously removed and to remove the modified interactiveelement. In another aspect, the method may further comprise receivinginput at the modified interactive element, wherein the inputcorresponding to a request to remove the modified interactive element,and modifying the graphical interface to remove the modified interactiveelement. In another aspect, the method may further comprise receivinginput corresponding to an interaction with the interactive element,wherein the input is associated with an adjustment for controllingoperation of the device, and transmitting the adjustment, whereinoperation of the device is controlled using the transmitted adjustment.

Alternative embodiments of the present invention may include a system,comprising one or more data processors, and a non-transitorycomputer-readable storage medium containing instructions, which whenexecuted on the one or more data processors, cause the one or moreprocessors to perform operations. The operations may include determininga current status associated with a device on a network, wherein thecurrent status indicates availability of the device on the network,generating a graphical interface, wherein the graphical interfaceincludes an interactive element corresponding to the device, and whereinthe interactive element indicates the current status associated with thedevice, displaying the graphical interface, determining a change to thecurrent status, wherein determining the change includes determining atype of status corresponding to the change, determining an updatedstatus associated with the device, determining that the device isassociated with the updated status for a threshold time period, andmodifying the graphical interface according to the type of status whenthe threshold time period is reached, wherein the modified graphicalinterface includes a modified interactive element corresponding to thedevice, and wherein the modified interactive element indicates theupdated status associated with the device.

Alternative embodiments of the present invention may include acomputer-program product tangibly embodied in a non-transitorymachine-readable storage medium, including instructions configured tocause a data processing apparatus to determine a current statusassociated with a device on a network, wherein the current statusindicates availability of the device on the network, generate agraphical interface, wherein the graphical interface includes aninteractive element corresponding to the device, and wherein theinteractive element indicates the current status associated with thedevice, display the graphical interface, determine a change to thecurrent status, wherein determining the change includes determining atype of status corresponding to the change, determine an updated statusassociated with the device, determine that the device is associated withthe updated status for a threshold time period, and modify the graphicalinterface according to the type of status when the threshold time periodis reached, wherein the modified graphical interface includes a modifiedinteractive element corresponding to the device, and wherein themodified interactive element indicates the updated status associatedwith the device.

This summary is not intended to identify key or essential features ofthe claimed subject matter, nor is it intended to be used in isolationto determine the scope of the claimed subject matter. The subject mattershould be understood by reference to appropriate portions of the entirespecification of this patent, any or all drawings, and each claim.

The foregoing, together with other features and embodiments, will becomemore apparent upon referring to the following specification, claims, andaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Illustrative embodiments of the present invention are described indetail below with reference to the following drawing figures:

FIG. 1 is an illustration of an example of a network environment, inaccordance with some embodiments.

FIG. 2 is a flowchart illustrating an embodiment of a process forregistering one or more network devices, in accordance with someembodiments.

FIG. 3 is an illustration of an example of a network environment, inaccordance with some embodiments.

FIG. 4 is an illustration of an example of a network environment, inaccordance with some embodiments.

FIG. 5 is an illustration of an example of a network environment, inaccordance with some embodiments.

FIG. 6 is an illustration of an example of a front view of a networkdevice, in accordance with an embodiment.

FIG. 7 is an illustration of an example of a side view of a networkdevice, in accordance with an embodiment.

FIG. 8 is an example of a block diagram of a network device, inaccordance with an embodiment.

FIG. 9 is a schematic illustration of a local area network including anetwork device that includes an appliance, in accordance with anembodiment.

FIG. 10 is an example of a block diagram of a network device includingan interface device attached to an appliance, in accordance with anembodiment.

FIG. 11 is an illustration of an example of a network environment, inaccordance with some embodiments.

FIG. 12 is an illustration of an example of a network environment, inaccordance with some embodiments.

FIG. 13 is an illustration of data flows for providing resource bundlesdefining network device interfaces within an example wireless networkenvironment, in accordance with some embodiments.

FIG. 14 is a flowchart illustrating a process for generating interfacemodules for network device interfaces, in accordance with someembodiments.

FIG. 15 is a flowchart illustrating a process for providing a resourcebundle for a network device, in accordance with some embodiments.

FIG. 16 is a flowchart illustrating a process for using resource bundlesto generate interface modules for network devices, in accordance withsome embodiments.

FIGS. 17-19 depict example interfaces for controlling network devices,in accordance with some embodiments.

FIG. 20 is an illustration of an interface for displaying networkdevices in accordance with some embodiments.

FIG. 21 is an illustration of an interface for displaying rules forcontrolling operation of network devices in accordance with someembodiments.

FIG. 22 shows a block diagram of a computing device for cachinginformation associated with network devices in accordance with someembodiments.

FIGS. 23 and 24 illustrate sequence diagrams of processes of cachinginformation associated with network devices in accordance with someembodiments.

FIG. 25 shows flowchart illustrating a process for caching informationin accordance with some embodiments.

FIG. 26 shows an embodiment of a process for providing a visualinterface module for controlling a network device in a wireless networkaccording to some embodiments.

FIG. 27A is an illustration of an example of a network environment, inaccordance with some embodiments.

FIG. 27B is an illustration of an example of a network environment, inaccordance with some embodiments.

FIG. 28 illustrates examples of information including criteria forinitiating actions according to some embodiments.

FIG. 29 illustrates examples of information including criteria forinitiating actions according to some embodiments.

FIGS. 30-35 depict examples of various interfaces for initiating and/orconfiguring actions in accordance with some embodiments.

FIG. 36 is a flowchart illustrating a process for initiating actionsusing an interface configured to control operation of a network devicein accordance with some embodiments.

FIG. 37 is a flowchart illustrating a process for initiating actionsusing an interface configured to control operation of a network devicein accordance with some embodiments.

FIG. 38 is a flowchart illustrating a process for initiating actionsusing an interface configured to control operation of a network devicein accordance with some embodiments.

FIG. 39 is an illustration of another example of a network environment,in accordance with some embodiments.

FIG. 40 is an illustration of data flows for providing meta-informationand control capabilities of devices connected to a gateway device withinan example network environment, in accordance with some embodiments.

FIG. 41 depicts an example of a gateway device interface, in accordancewith some embodiments.

FIG. 42 depicts an example of a proprietary interface of a clientdevice, in accordance with some embodiments.

FIG. 43 depicts an example of a proprietary interface of a clientdevice, in accordance with some embodiments.

FIG. 44 depicts an example of a proprietary interface of a clientdevice, in accordance with some embodiments.

FIG. 45 is a flowchart illustrating an embodiment of a process ofdetecting a device connected to a gateway device and determining controlcapabilities and meta-information of the connected device, in accordancewith some embodiments.

FIG. 46 shows an embodiment of a process for providing a visualinterface module for controlling a network device in a wireless networkaccording to some embodiments.

FIG. 47 is an illustration of an example of a network environment, inaccordance with some embodiments.

FIGS. 48-53 depict examples of various interfaces for displaying astatus associated with network devices in accordance with someembodiments.

FIG. 54 is a flowchart illustrating a process for displaying a statusassociated with network devices in accordance with some embodiments.

FIG. 55 is an illustration of an example of a proprietary interface of adevice, in accordance with some embodiments.

FIG. 56 is an illustration of example proprietary interfaces of adevice, in accordance with some embodiments.

FIG. 57 is shows an embodiment of a process for displaying tiles basedon data and rankings, in accordance with some embodiments.

FIGS. 58-60 show data structures for managing operations of networkdevices, in accordance with some embodiments.

FIGS. 61 and 62 depict processes for controlling operations of networkdevices, in accordance with some embodiments.

DETAILED DESCRIPTION

In the following description, for the purposes of explanation, specificdetails are set forth in order to provide a thorough understanding ofembodiments of the invention. However, it will be apparent that variousembodiments may be practiced without these specific details. The figuresand description are not intended to be restrictive.

The ensuing description provides exemplary embodiments only, and is notintended to limit the scope, applicability, or configuration of thedisclosure. Rather, the ensuing description of the exemplary embodimentswill provide those skilled in the art with an enabling description forimplementing an exemplary embodiment. It should be understood thatvarious changes may be made in the function and arrangement of elementswithout departing from the spirit and scope of the invention as setforth in the appended claims.

Specific details are given in the following description to provide athorough understanding of the embodiments. However, it will beunderstood by one of ordinary skill in the art that the embodiments maybe practiced without these specific details. For example, circuits,systems, networks, processes, and other components may be shown ascomponents in block diagram form in order not to obscure the embodimentsin unnecessary detail. In other instances, well-known circuits,processes, algorithms, structures, and techniques may be shown withoutunnecessary detail in order to avoid obscuring the embodiments.

Also, it is noted that individual embodiments may be described as aprocess which is depicted as a flowchart, a flow diagram, a data flowdiagram, a structure diagram, or a block diagram. Although a flowchartmay describe the operations as a sequential process, many of theoperations can be performed in parallel or concurrently. In addition,the order of the operations may be re-arranged. A process is terminatedwhen its operations are completed, but could have additional steps notincluded in a figure. A process may correspond to a method, a function,a procedure, a subroutine, a subprogram, etc. When a process correspondsto a function, its termination can correspond to a return of thefunction to the calling function or the main function.

The term “machine-readable storage medium” or “computer-readable storagemedium” includes, but is not limited to, portable or non-portablestorage devices, optical storage devices, and various other mediumscapable of storing, containing, or carrying instruction(s) and/or data.A machine-readable storage medium or computer-readable storage mediummay include a non-transitory medium in which data can be stored and thatdoes not include carrier waves and/or transitory electronic signalspropagating wirelessly or over wired connections. Examples of anon-transitory medium may include, but are not limited to, a magneticdisk or tape, optical storage media such as compact disk (CD) or digitalversatile disk (DVD), flash memory, memory or memory devices. Acomputer-program product may include code and/or machine-executableinstructions that may represent a procedure, a function, a subprogram, aprogram, a routine, a subroutine, a module, a software package, a class,or any combination of instructions, data structures, or programstatements. A code segment may be coupled to another code segment or ahardware circuit by passing and/or receiving information, data,arguments, parameters, or memory contents. Information, arguments,parameters, data, etc. may be passed, forwarded, or transmitted via anysuitable means including memory sharing, message passing, token passing,network transmission, etc.

Furthermore, embodiments may be implemented by hardware, software,firmware, middleware, microcode, hardware description languages, or anycombination thereof. When implemented in software, firmware, middlewareor microcode, the program code or code segments to perform the necessarytasks (e.g., a computer-program product) may be stored in amachine-readable medium. A processor(s) may perform the necessary tasks.

Systems depicted in some of the figures may be provided in variousconfigurations. In some embodiments, the systems may be configured as adistributed system where one or more components of the system aredistributed across one or more networks in a cloud computing system.

A network may be set up to provide an access device user with access tovarious devices connected to the network. For example, a network mayinclude one or more network devices that provide a user with the abilityto remotely configure or control the network devices themselves or oneor more electronic devices (e.g., appliances) connected to the networkdevices. The electronic devices may be located within an environment ora venue that can support the network. An environment can include, forexample, a home, an office, a business, an automobile, a park, or thelike. A network may include one or more gateways that allow clientdevices (e.g., network devices, access devices, or the like) to accessthe network by providing wired connections and/or wireless connectionsusing radio frequency channels in one or more frequency bands. The oneor more gateways may also provide the client devices with access to oneor more external networks, such as a cloud network, the Internet, and/orother wide area networks.

A local area network, such as a user's home local area network, caninclude multiple network devices that provide various functionalities.Network devices may be accessed and controlled using an access deviceand/or one or more network gateways. One or more gateways in the localarea network may be designated as a primary gateway that provides thelocal area network with access to an external network. The local areanetwork can also extend outside of the user's home and may includenetwork devices located outside of the user's home. For instance, thelocal area network can include network devices such as exterior motionsensors, exterior lighting (e.g., porch lights, walkway lights, securitylights, or the like), garage door openers, sprinkler systems, or othernetwork devices that are exterior to the user's home. It is desirablefor a user to be able to access the network devices while located withinthe local area network and also while located remotely from the localarea network. For example, a user may access the network devices usingan access device within the local area network or remotely from thelocal area network.

In some embodiments, a user may create an account with login informationthat is used to authenticate the user and allow access to the networkdevices. For example, once an account is created, a user may enter thelogin information in order to access a network device in a logicalnetwork.

In some embodiments, an accountless authentication process may beperformed so that the user can access one or more network devices withina logical network without having to enter network device logincredentials each time access is requested. While located locally withinthe local area network, an access device may be authenticated based onthe access device's authentication with the logical network. Forexample, if the access device has authorized access to the logicalnetwork (e.g., a WiFi network provided by a gateway), the networkdevices paired with that logical network may allow the access device toconnect to them without requiring a login. Accordingly, only users ofaccess devices that have authorization to access the logical network areauthorized to access network devices within the logical network, andthese users are authorized without having to provide login credentialsfor the network devices.

An accountless authentication process may also be performed when theuser is remote so that the user can access network devices within thelogical network, using an access device, without having to enter networkdevice login credentials. While remote, the access device may access thenetwork devices in the local area network using an external network,such as a cloud network, the Internet, or the like. One or more gatewaysmay provide the network devices and/or access device connected to thelocal area network with access to the external network. To allowaccountless authentication, a cloud network server may provide a networkID and/or one or more keys to a network device and/or to the accessdevice (e.g., running an application, program, or the like). In somecases, a unique key may be generated for the network device and aseparate unique key may be generated for the access device. The keys maybe specifically encrypted with unique information identifiable only tothe network device and the access device. The network device and theaccess device may be authenticated using the network ID and/or eachdevice's corresponding key each time the network device or access deviceattempts to access the cloud network server.

In some embodiments, a home local area network may include a singlegateway, such as a router. A network device within the local areanetwork may pair with or connect to the gateway and may obtaincredentials from the gateway. For example, when the network device ispowered on, a list of gateways that are detected by the network devicemay be displayed on an access device (e.g., via an application, program,or the like installed on and executed by the access device). In thisexample, only the single gateway is included in the home local areanetwork (e.g., any other displayed gateways may be part of other localarea networks). In some embodiments, only the single gateway may bedisplayed (e.g., when only the single gateway is detected by the networkdevice). A user may select the single gateway as the gateway with whichthe network device is to pair and may enter login information foraccessing the gateway. The login information may be the same informationthat was originally set up for accessing the gateway (e.g., a networkuser name and password, a network security key, or any other appropriatelogin information). The access device may send the login information tothe network device and the network device may use the login informationto pair with the gateway. The network device may then obtain thecredentials from the gateway. The credentials may include a service setidentification (SSID) of the home local area network, a media accesscontrol (MAC) address of the gateway, and/or the like. The networkdevice may transmit the credentials to a server of a wide area network,such as a cloud network server. In some embodiments, the network devicemay also send to the server information relating to the network device(e.g., MAC address, serial number, or the like) and/or informationrelating to the access device (e.g., MAC address, serial number,application unique identifier, or the like).

The cloud network server may register the gateway as a logical networkand may assign the first logical network a network identifier (ID). Thecloud network server may further generate a set of security keys, whichmay include one or more security keys. For example, the server maygenerate a unique key for the network device and a separate unique keyfor the access device. The server may associate the network device andthe access device with the logical network by storing the network ID andthe set of security keys in a record or profile. The cloud networkserver may then transmit the network ID and the set of security keys tothe network device. The network device may store the network ID and itsunique security key. The network device may also send the network ID andthe access device's unique security key to the access device. In someembodiments, the server may transmit the network ID and the accessdevice's security key directly to the access device. The network deviceand the access device may then communicate with the cloud server usingthe network ID and the unique key generated for each device.Accordingly, the access device may perform accountless authentication toallow the user to remotely access the network device via the cloudnetwork without logging in each time access is requested. Also, thenetwork device can communicate with the server regarding the logicalnetwork.

In some embodiments, a local area network may include multiple gateways(e.g., a router and a range extender) and multiple network devices. Forexample, a local area network may include a first gateway paired with afirst network device, and a second gateway paired with a second networkdevice. In the event credentials for each gateway are used to create alogical network, a server (e.g., a cloud network server) may registerthe first gateway as a first logical network and may register the secondgateway as a second logical network. The server may generate a firstnetwork ID and a first set of security keys for the first logicalnetwork. The first set of security keys may include a unique securitykey for the first network device and a unique security key for theaccess device for use in accessing the first network device on the firstlogical network. The server may register the second gateway as thesecond logical network due to differences in the credentials between thefirst gateway and second gateway. The server may assign the secondgateway a second network ID and may generate a second set of securitykeys. For example, the server may generate a unique security key for thesecond network device and may generate a unique security key for theaccess device for use in accessing the second network device on thesecond logical network. The server may associate the first networkdevice and the access device with the first logical network by storingthe first network ID and the first set of security keys in a firstrecord or profile. The server may also associate the second networkdevice and the access device with the second logical network by storingthe second network ID and the second set of security keys in a record orprofile. The server may then transmit the first network ID and the firstset of security keys to the first network device, and may transmit thesecond network ID and the second set of security keys to the secondnetwork device. The two network devices may store the respective networkID and set of security keys of the gateway with which each networkdevice is connected. Each network device may send the respective networkID and the access device's unique security key to the access device. Thenetwork devices and the access device may then communicate with thecloud server using the respective network ID and the unique keygenerated for each device.

Accordingly, when multiple gateways are included in the home local areanetwork, multiple logical networks associated with different networkidentifiers may be generated for the local area network. When the accessdevice is located within range of both gateways in the local areanetwork, there is no problem accessing both network devices due to theability of the access device to perform local discovery techniques(e.g., universal plug and play (UPnP)). However, when the user islocated remotely from the local area network, the access device may onlybe associated with one logical network at a time, which prevents theaccess device from accessing network devices of other logical networkswithin the local area network.

FIG. 1 illustrates an example of a local area network 100. The localarea network 100 includes network device 102, network device 104, andnetwork device 106. In some embodiments, any of the network devices 102,104, 106 may include an Internet of Things (IoT) device. As used herein,an IoT device is a device that includes sensing and/or controlfunctionality as well as a WiFi™ transceiver radio or interface, aBluetooth™ transceiver radio or interface, a Zigbee™ transceiver radioor interface, an Ultra-Wideband (UWB) transceiver radio or interface, aWiFi-Direct transceiver radio or interface, a Bluetooth™ Low Energy(BLE) transceiver radio or interface, an infrared (IR) transceiver,and/or any other wireless network transceiver radio or interface thatallows the IoT device to communicate with a wide area network and withone or more other devices. In some embodiments, an IoT device does notinclude a cellular network transceiver radio or interface, and thus maynot be configured to directly communicate with a cellular network. Insome embodiments, an IoT device may include a cellular transceiverradio, and may be configured to communicate with a cellular networkusing the cellular network transceiver radio. The network devices 102,104, 106, as IoT devices or other devices, may include home automationnetwork devices that allow a user to access, control, and/or configurevarious home appliances located within the user's home (e.g., atelevision, radio, light, fan, humidifier, sensor, microwave, iron,and/or the like), or outside of the user's home (e.g., exterior motionsensors, exterior lighting, garage door openers, sprinkler systems, orthe like). For example, network device 102 may include a home automationswitch that may be coupled with a home appliance. In some embodiments,network devices 102, 104, 106 may be used in other environments, such asa business, a school, an establishment, a park, or any place that cansupport the local area network 100 to enable communication with networkdevices 102, 104, 106. For example, a network device can allow a user toaccess, control, and/or configure devices, such as office-relateddevices (e.g., copy machine, printer, fax machine, or the like), audioand/or video related devices (e.g., a receiver, a speaker, a projector,a DVD player, a television, or the like), media-playback devices (e.g.,a compact disc player, a CD player, or the like), computing devices(e.g., a home computer, a laptop computer, a tablet, a personal digitalassistant (PDA), a computing device, a wearable device, or the like),lighting devices (e.g., a lamp, recessed lighting, or the like), devicesassociated with a security system, devices associated with an alarmsystem, devices that can be operated in an automobile (e.g., radiodevices, navigation devices), and/or the like.

A user may communicate with the network devices 102, 104, 106 using anaccess device 108. The access device 108 may include anyhuman-to-machine interface with network connection capability thatallows access to a network. For example, the access device 108 mayinclude a stand-alone interface (e.g., a cellular telephone, asmartphone, a home computer, a laptop computer, a tablet, a personaldigital assistant (PDA), a computing device, a wearable device such as asmart watch, a wall panel, a keypad, or the like), an interface that isbuilt into an appliance or other device e.g., a television, arefrigerator, a security system, a game console, a browser, or thelike), a speech or gesture interface (e.g., a Kinect™ sensor, aWiimote™, or the like), an IoT device interface (e.g., an Internetenabled device such as a wall switch, a control interface, or othersuitable interface), or the like. In some embodiments, the access device108 may include a cellular or other broadband network transceiver radioor interface, and may be configured to communicate with a cellular orother broadband network using the cellular or broadband networktransceiver radio. In some embodiments, the access device 108 may notinclude a cellular network transceiver radio or interface. While only asingle access device 108 is shown in FIG. 1, one of ordinary skill inthe art will appreciate that multiple access devices may communicatewith the network devices 102, 104, 106. The user may interact with thenetwork devices 102, 104, or 106 using an application, a web browser, aproprietary program, or any other program executed and operated by theaccess device 108. In some embodiments, the access device 108 maycommunicate directly with the network devices 102, 104, 106 (e.g.,communication signal 116). For example, the access device 108 maycommunicate directly with network device 102, 104, 106 using Zigbee™signals, Bluetooth™ signals, WiFi™ signals, infrared (IR) signals, UWBsignals, WiFi-Direct signals, BLE signals, sound frequency signals, orthe like. In some embodiments, the access device 108 may communicatewith the network devices 102, 104, 106 via the gateways 110, 112 (e.g.,communication signal 118) and/or the cloud network 114 (e.g.,communication signal 120).

The local area network 100 may include a wireless network, a wirednetwork, or a combination of a wired and wireless network. A wirelessnetwork may include any wireless interface or combination of wirelessinterfaces (e.g., Zigbee™, Bluetooth™, WiFi™, IR, UWB, WiFi-Direct, BLE,cellular, Long-Term Evolution (LTE), WiMax™, or the like). A wirednetwork may include any wired interface (e.g., fiber, ethernet,powerline ethernet, ethernet over coaxial cable, digital signal line(DSL), or the like). The wired and/or wireless networks may beimplemented using various routers, access points, bridges, gateways, orthe like, to connect devices in the local area network 100. For example,the local area network may include gateway 110 and gateway 112. Gateway110 or 112 can provide communication capabilities to network devices102, 104, 106 and/or access device 108 via radio signals in order toprovide communication, location, and/or other services to the devices.The gateway 110 is directly connected to the external network 114 andmay provide other gateways and devices in the local area network withaccess to the external network 114. The gateway 110 may be designated asa primary gateway. While two gateways 110 and 112 are shown in FIG. 1,one of ordinary skill in the art will appreciate that any number ofgateways may be present within the local area network 100.

The network access provided by gateway 110 and gateway 112 may be of anytype of network familiar to those skilled in the art that can supportdata communications using any of a variety of commercially-availableprotocols. For example, gateways 110, 112 may provide wirelesscommunication capabilities for the local area network 100 usingparticular communications protocols, such as WiFi™ (e.g., IEEE 802.11family standards, or other wireless communication technologies, or anycombination thereof). Using the communications protocol(s), the gateways110, 112 may provide radio frequencies on which wireless enabled devicesin the local area network 100 can communicate. A gateway may also bereferred to as a base station, an access point, Node B, Evolved Node B(eNodeB), access point base station, a Femtocell, home base station,home Node B, home eNodeB, or the like.

The gateways 110, 112 may include a router, a modem, a range extendingdevice, and/or any other device that provides network access among oneor more computing devices and/or external networks. For example, gateway110 may include a router or access point, and gateway 112 may include arange extending device. Examples of range extending devices may includea wireless range extender, a wireless repeater, or the like.

A router gateway may include access point and router functionality, andmay further include an Ethernet switch and/or a modem. For example, arouter gateway may receive and forward data packets among differentnetworks. When a data packet is received, the router gateway may readidentification information (e.g., a media access control (MAC) address)in the packet to determine the intended destination for the packet. Therouter gateway may then access information in a routing table or routingpolicy, and may direct the packet to the next network or device in thetransmission path of the packet. The data packet may be forwarded fromone gateway to another through the computer networks until the packet isreceived at the intended destination.

A range extending gateway may be used to improve signal range andstrength within a local area network. The range extending gateway mayreceive an existing signal from a router gateway or other gateway andmay rebroadcast the signal to create an additional logical network. Forexample, a range extending gateway may extend the network coverage ofthe router gateway when two or more devices on the local area networkneed to be connected with one another, but the distance between one ofthe devices and the router gateway is too far for a connection to beestablished using the resources from the router gateway. As a result,devices outside of the coverage area of the router gateway may be ableto connect through the repeated network provided by the range extendinggateway. The router gateway and range extending gateway may exchangeinformation about destination addresses using a dynamic routingprotocol.

The gateways 110 and 112 may also provide the access device 108 and thenetwork devices 102, 104, 106 with access to one or more externalnetworks, such as the cloud network 114, the Internet, and/or other widearea networks. In some embodiments, the network devices 102, 104, 106may connect directly to the cloud network 114, for example, usingbroadband network access such as a cellular network. The cloud network114 may include a cloud infrastructure system that provides cloudservices. In certain embodiments, services provided by the cloud network114 may include a host of services that are made available to users ofthe cloud infrastructure system on demand, such as registration andaccess control of network devices 102, 104, 106. Services provided bythe cloud infrastructure system can dynamically scale to meet the needsof its users. The cloud network 114 may comprise one or more computers,servers, and/or systems. In some embodiments, the computers, servers,and/or systems that make up the cloud network 114 are different from theuser's own on-premises computers, servers, and/or systems. For example,the cloud network 114 may host an application, and a user may, via acommunication network such as the Internet, on demand, order and use theapplication.

In some embodiments, the cloud network 114 may host a Network AddressTranslation (NAT) Traversal application in order to establish a secureconnection between the cloud network 114 and one or more of the networkdevices 102, 104, 106. For example, a separate secure TransmissionControl Protocol (TCP) connection may be established by each networkdevice 102, 104, 106 for communicating between each network device 102,104, 106 and the cloud network 114. In some embodiments, each secureconnection may be kept open for an indefinite period of time so that thecloud network 114 can initiate communications with each respectivenetwork device 102, 104, or 106 at any time. In some cases, other typesof communications between the cloud network 114 and the network devices102, 104, 106 and/or the access device 108 may be supported using othertypes of communication protocols, such as a Hypertext Transfer Protocol(HTTP) protocol, a Hypertext Transfer Protocol Secure (HTTPS) protocol,or the like. In some embodiments, communications initiated by the cloudnetwork 114 may be conducted over the TCP connection, and communicationsinitiated by a network device may be conducted over a HTTP or HTTPSconnection. In certain embodiments, the cloud network 114 may include asuite of applications, middleware, and database service offerings thatare delivered to a customer in a self-service, subscription-based,elastically scalable, reliable, highly available, and secure manner.

It should be appreciated that the local area network 100 may have othercomponents than those depicted. Further, the embodiment shown in thefigure is only one example of a local area network that may incorporatean embodiment of the invention. In some other embodiments, local areanetwork 100 may have more or fewer components than shown in the figure,may combine two or more components, or may have a differentconfiguration or arrangement of components.

Upon being powered on or reset, the network devices 102, 104, 106 may beregistered with the cloud network 114 and associated with a logicalnetwork within the local area network 100. FIG. 2 illustrates an exampleof a process 200 for registering one or more network devices, such asthe network devices 102, 104, 106 illustrated in FIG. 1. When multiplenetwork devices 102, 104, 106 and gateways 110, 112 are included withina local area network, the network devices and/or gateways may beinstalled at different times, resulting in the techniques described withrespect to FIG. 2 possibly occurring for each network device and/orgateway at different points in time. For example, a user may installnetwork device 102 at a first point in time on a first floor of theuser's house. Gateway 110 may also be located on the first floor,resulting in the network device 102 pairing with gateway 110. The usermay later install gateway 112 and network device 106 on a second floorof the user's home, resulting in the network device 106 pairing withgateway 112.

At 202, a network device may detect one or more gateways upon beingpowered on or reset. In some embodiments, a provisioning process mayoccur when the network device is powered on or reset and detected by anaccess device (e.g., access device 108). During the provisioningprocess, the access device may directly communicate with the networkdevice. In some embodiments, direct communication between networkdevices (e.g., network devices 102, 104, 106) and access device (e.g.,access device 108) may occur using various communications protocols,such as Universal Plug and Play (UPnP), Bluetooth®, Zigbee®,Ultra-Wideband (UWB), WiFi-Direct, WiFi, Bluetooth® Low Energy (BLE),sound frequencies, and/or the like.

The provisioning process may include pairing the network device with agateway and registering the gateway, network device, and access devicewith a server, such as a server located within the cloud network 114.For example, upon being powered on or reset to factory settings, thenetwork device may send or broadcast identification information to oneor more access devices. The identification information may be sentduring a discovery process. For example, the identification informationmay be sent in response to a discovery request from an access device. Insome cases, the identification information may include a name of thenetwork device.

An application, program, or the like that is installed on and executedby the access device may receive the identification information from thenetwork device. When the application on the access device is launched bya user, the access device may display the identification information forselection by the user. Once the network device identificationinformation is selected, the access device may send a signal to thenetwork device indicating that it has been selected. The network devicemay then send to the access device a list of gateways that are detectedby the network device. The access device may receive and display thelist of gateways. In some embodiments, the list of gateways includesmultiple gateways (e.g., gateways 110 and 112) that are located withinthe local area network. The user may select the gateway that the userwishes for the network device to pair. For example, the gateway thatprovides the best signal strength for the network device may beselected. The access device may then prompt the user to enter logininformation that is required for accessing the network signals providedby the selected gateway. For example, the login information may be thesame information that was originally set up to access the gatewaynetwork signals (e.g., when the gateway was initially installed). Onceentered, the access device may send the login information to the networkdevice. The network device may use the login information to pair withthe selected gateway. As one example, network device 102 and networkdevice 104 may be paired with gateway 110, and network device 106 may bepaired with gateway 112.

Once paired with a gateway, the network device may be registered with acloud network (e.g., cloud network 114). For example, the access device(e.g., via the application, program, or the like) may instruct thenetwork device to register with the cloud network upon receivingconfirmation from the network device that it has been successfullypaired with a gateway. At 204, the network device may obtain credentialsfrom the gateway as part of the registration process. For example,network device 102 may obtain credentials from gateway 110. At a same orlater point in time, network devices 104 and 106 may obtain credentialsfrom gateways 110 and 112, respectively. In some embodiments, thecredentials may include a SSID of the local area network and a MACaddress of the gateway. An SSID received from two gateways (e.g.,gateways 110, 112) may be the same due to the gateways both being withinthe same local area network. In some cases, the SSID of the two gatewaysmay be different. The MAC address of each of the gateways may be uniqueto each gateway. As a result of each gateway having a unique MACaddress, the credentials obtained from a gateway may be unique to thatparticular gateway. One of ordinary skill in the art will appreciatethat other credentials may be obtained from a gateway, such as anInternet Protocol address, or the like.

The network device may then send the gateway credentials to the cloudnetwork at 206. For example, the network devices 102, 104, 106 may sendcredentials for the gateway with which each is paired to the serverlocated within the cloud network 114. For example, network device 102may transmit the credentials obtained from gateway 110 to the server,and network device 106 may transmit the credentials obtained fromgateway 112 to the server. In some embodiments, the network device mayalso send information relating to the network device (e.g., MAC address,serial number, make, model number, firmware version, and/or an interfacemodule identifier, or the like) to the server, and/or informationrelating to the access device (e.g., MAC address, serial number,application unique identifier, or the like) to the server. In someembodiments, the communication of the credentials, the network deviceinformation, and/or the access device information sent from the networkdevice to the cloud network server may be in a Hypertext TransferProtocol (HTTP) format, a Hypertext Transfer Protocol Secure (HTTPS)format, a secure Transmission Control Protocol (TCP) format, or thelike. One of ordinary skill in the art will appreciate that othercommunication formats may be used to communicate between the networkdevice and the cloud network server.

Once the credentials, network device information, and/or access deviceinformation are received by the server, the server may register eachgateway as a logical network within the local area network and maygenerate a network ID for each logical network. For example, the servermay register the gateway 110 as a first logical network. During theregistration process, the server may generate a first network ID foridentifying the first logical network. As noted above, one of ordinaryskill in the art will appreciate that any number of gateways may bepresent within the local area network, and thus that any number oflogical networks may be registered for the local area network. Theserver may further generate a first set of security keys forauthenticating the network device and the access device. For example,the server may generate a unique key for the network device 102 and aseparate unique key for the access device 108.

In some embodiments, as previously described, network device 104 mayalso be paired with gateway 110 at the same or a later point in time asthe network device 102. During registration of the network device 104,the server may determine that the access device 108 has already beenregistered with another network device (e.g., network device 102) thatis associated with the same logical network of gateway 110. In suchembodiments, the server may retrieve the first network ID that was usedin registering the first logical network. The server may also generate anew unique security key for the network device 104, and may retrieve theunique key that was previously generated for the access device 108 whenregistering the gateway 110 as the first logical network.

The gateway 112 may also be registered by the server as a second logicalnetwork with a second network ID. A second set of security keys may begenerated for the network device 106 and the access device 108. Forexample, the server may generate a unique security key for the networkdevice 106 and a unique security key for the access device 108 as itrelates to the second logical network. In some embodiments, the gatewaymay 112 be installed at a later point in time after the gateway 110 isinstalled, and thus may be registered as the second logical network atthe later point in time.

A record or profile may then be created for associating each network IDwith the credentials of a corresponding gateway, the correspondingnetwork device(s), and the access device. For example, the server of thecloud network 114 may associate the first network ID with thecredentials of gateway 110. Similarly, the server may associate thesecond network ID with the credentials of gateway 112. In someembodiments, the server performs the association by generating andstoring a record including the network ID, the set of security keys, thegateway credentials, the network devices associated with the network ID(e.g., MAC address or serial number of a network device), the accessdevices associated with the network ID (e.g., MAC address, serialnumber, application unique identifier, or the like), and/or any otherinformation relevant to the network devices and/or gateways. Forexample, the server may store the first network ID and the first set ofsecurity keys in a first record at a first memory space (e.g., in Flash,DRAM, a database, or the like) along with the SSID and MAC address forgateway 110 and an identifier of the network devices 102 and/or 104. Theserver may also store the second network ID and the second set ofsecurity keys in a second record at a second memory space along with theSSID and MAC address for gateway 112 and an identifier of the networkdevice 106. In some embodiments, an example of a network deviceidentifier may include a MAC address of the network device, a serialnumber of the network device, or any other unique identifier.

Each of the first and second network IDs may include a unique number oralphanumeric string generated sequentially or randomly. For example, thefirst time a network device and an associated gateway are registered onthe cloud network 114, the unique network ID for the logical network ofthe gateway may start with 7000000. Each subsequent logical network thatis created may be a sequential increment of the initial network ID(e.g., 7000001, 7000002, 7000003, etc.). As another example, the networkID may be generated by a random or pseudo-random number generator. Oneof ordinary skill in the art will appreciate that other techniques forgenerating a unique ID may be used. The technique used to generate thenetwork IDs may be dependent on a type of database that is included inthe cloud network 114. For example, different databases may havedifferent proprietary mechanisms for creating a unique identifier.

The set of keys generated for each logical network may be generatedusing database specific technique. For example, a MySQL technique may beused to generate the sets of keys. Each key may include a universallyunique identifier (UUID) or a globally unique identifier (GUID). Asdescribed above, for each logical network, the server may generate aunique key for a network device and a separate unique key for an accessdevice.

At 208, the network device may receive the network ID and the set ofsecurity keys. For example, once the server has generated a record orprofile associating the network device 102 with the first logicalnetwork, the server may transmit the first network ID and the first setof security keys to the network device 102. The network device 102 maystore the first network ID and one or more keys of the first set ofkeys. For example, the network device 102 may store the unique securitykey that was created by the server for the network device 102.

As noted previously, the network devices 102, 104, 106 and gateways 110,112 may be installed at different times. For example, in someembodiments, network device 104 may be installed at a point in timeafter the first logical network is created based on the pairing betweengateway 110 and network device 102. In such embodiments, upon beingpowered on, the network device 104 may pair with gateway 110, obtaincredentials from gateway 110, and transmit the credentials to the serverin the cloud network 114 using similar techniques as those describedabove. The server may associate the network device 104 with thepreviously generated first network ID. As described above, the servermay also generate a new unique security key for the network device 104,and may retrieve the unique key that was previously generated for theaccess device 108 when registering the first logical network. Thenetwork device 104 may then receive and store the first network ID andthe security keys from the server.

At 210, the network device may send the network ID and the set ofsecurity keys to the access device. For example, the network device 102may send to the access device 108 the first network ID and the uniquesecurity key generated for the access device 108. The network device 102and the access device 108 may then communicate with the cloud networkserver using the first network ID and each device's unique key. In someembodiments, the network device and the access device may generate asignature using their respective security key. The signature is sent tothe cloud network server along with a communication from the networkdevice or access device. The cloud network server may process thesignature in order to authenticate each device, as described below. Thenetwork device and access device may use different techniques togenerate a signature.

A network device may generate a signature using its uniquely generatedsecurity key. For example, the signature may be expressed as:Authorization=MacAddress“:”Signature“:”ExpirationTime. The Authorizationterm may be an attribute, and the MacAddress, Signature, andExpirationTime terms may include values for the Authorization attribute.In particular, the MacAddress value may include the MAC address of thenetwork device, which may include a unique alphanumeric or numericstring. The network device may retrieve its MAC address from memory andplace it in the MacAddress field. The Signature value may be expressedas: Signature=Base64(HMAC-SHA1(PrivateKey, StringToSign)). The Signaturevalue may include an alphanumeric or numeric string. HMAC-SHA1 is anopen source technique that includes a Hash-based Message AuthenticationCode (HMAC) using a SHA1 hash function. The HMAC-SHA1 technique uses thevalues PrivateKey and StringToSign as inputs. The PrivateKey inputincludes the unique security key that was generated by the server forthe network device. The StringToSign input may be expressed asStringToSign=MacAddress+“\n”+SerialNumber+“\n”+ExpirationTime.Accordingly, the StringToSign input is generated by appending a serialnumber of the network device and an expiration time to the networkdevice's MAC address. The ExpirationTime term may indicate the period oftime for which the signature is valid. In some embodiments, theExpirationTime term may include a current time at which the signature isgenerated plus period of time for which the signature is valid. In oneexample, the ExpirationTime term may be expressed asExpirationTime=Number of seconds since Jan. 1, 1970.

The network device may place the signature in a data packet fortransmission with a communication signal to the cloud network server.The network device may also place the network ID in the data packet. Thesignature and the network ID, if included, may be used by the cloudnetwork server to verify that the network device is associated with thelogical network. In some embodiments, a signature is provided with eachcommunication sent from the network device to the server. Once thesignature is received by the server, the server generates a signatureusing the same expression as that used by the network device. Forexample, the server may retrieve the network device's key and otherrelevant information from storage and generate the signature using thekey and the other information using the expression described above. Theserver then verifies whether the signatures match. Upon determining thatthe signatures match, the server authenticates the network device'scommunication.

An access device may also generate a signature using its uniquelygenerated security key. For example, the access device signature may beexpressed as: Authorization=SDU UniqueId“:”Signature“:”ExpirationTime.The Authorization term may be an attribute, and the SDU UniqueId,Signature, and ExpirationTime terms may include values for theAuthorization attribute. The SDU UniqueId term may include a uniquephone identifier. The SDU UniqueId value may depend on the type ofaccess device that is used and the type of values that may be accessedand/or generated by the type of access device. In some cases, one typeof access device may not allow an application to access a uniqueidentifier of the access device (e.g., a serial number, UUID, or thelike). In such cases, the SDU UniqueId value may include a valuegenerated by an application or program installed on and executed on theaccess device that is used to access the network device. The value maybe unique to the application or program that generated the value. Inother cases, another type of access device may allow an application toaccess a unique identifier of the access device. In such cases, the SDUUniqueId value may include a value that is unique to the access deviceitself, such as a serial number, UUID, or the like. In this example, theaccess device may retrieve the unique value from storage within theaccess device. One of ordinary skill in the art will appreciate thatother unique identifiers may be used to uniquely identify the accessdevice. The Signature value may be expressed as:Signature=Base64(HMAC-SHA1(PrivateKey, StringToSign)). Using thisexpression, the input to the HMAC-SHA1 technique may include aPrivateKey term and a StringToSign term. The PrivateKey input includesthe unique security key that was generated by the server for the accessdevice with regard to a particular logical network. The StringToSigninput may be expressed as StringToSign=UniqueId+“\n”+“\n”+ExpirationTime. The StringToSign value is different from the StringToSign valuegenerated by network device in that no serial number is included.Accordingly, the StringToSign input is generated by appending anexpiration time to the access device's unique identifier. TheExpirationTime term may indicate the period of time for which thesignature is valid, similar to that above for the signature generated bythe network device.

The access device may place the signature in a data packet and maytransmit the data packet to the cloud network server with acommunication signal. The network device may also place the network IDin the data packet. The signature and the network ID, if included, maybe used by the cloud network server to verify that the access device isassociated with the logical network and authorized to communicate withone or more network devices associated with the logical network. In someembodiments, a signature is provided with each communication sent fromthe access device to the server. The cloud server may receive thesignature and may generate a signature using the same expression as thatused by the access device. For example, the server may retrieve theaccess device's key and other relevant information from storage andgenerate the signature using the key and the other information using theexpression described above. The server then verifies whether thesignatures match. Upon determining that the signatures match, the serverauthenticates the access device and allows it to communicate with one ormore of the network devices associated with logical network.

Once the provisioning process is completed, the access device 108 mayaccess the network device 102 locally via the gateway 110 (e.g.,communication signal 118) or remotely via the cloud network 114 (e.g.,communication signal 120). In some embodiments, the communicationbetween the access device 108 and the cloud network 114 may be a HTTP orHTTPS communication. One of ordinary skill in the art will appreciatethat other communication mechanisms may be used to communicate betweenthe access device 108 and the cloud network 114.

The network 100 may enable a user to monitor and/or control operation ofthe devices 102 and 104. For example, a user may monitor and/or controloperation of devices by interacting with a visual interface of thegateway 110 (i.e., a web page for gateway 110) and/or a visual interfacerendered on a display of an access device, such as access device 108. Insome embodiments, an application may be run on the access device. Theapplication may cause the access device to present a graphical interfacethat includes a visual interface for each device accessible on thenetwork 100.

A network device may generate and/or provide a “status” of the networkdevice. In certain embodiments, the status or state of a network devicecan be indicated on a visual interface on the access device, for examplewithin the tile with text and/or graphically. The status of the networkdevice can change based on time (e.g., a period, an interval, or othertime schedule). The status of a network device may be any piece ofinformation pertinent to that particular network device. The status of anetwork device may be any changeable variable of that particular networkdevice. For example, the status of a network device may include a stateof the network device itself (e.g., on or off) or how the network deviceis situated within the network with respect to the other network andother network devices throughout the network. For example, the status ofa network device may refer to the network device's proximity to anothernetwork device and/or its ability to communicate with another networkdevice because of the relative signal strength between the two networkdevices. In certain embodiments, the status can include a value or someother information indicating a unit of measure for a setting or anattribute related to operation of a device connected to the networkdevice. The setting or the attribute can be adjustable within a range ofvalues. For example, the device connected to the network device can be alight bulb and the status can include a value corresponding tobrightness (e.g., a percentage of total brightness) emitted by the lightbulb when the light bulb is powered-on. In another example, the devicecan be a motion sensor and the status can include a value correspondingto sensitivity of the sensor in a range of values between 0 to 100 whenthe sensor is powered on. In yet another example, the device can be afan and the status can include a value corresponding to a speed of thefan on a scale of 0 to 100 when the fan is powered-on.

As described above, upon being powered on or reset, the network devices102 and/or 104 may be registered with the cloud network 114 andassociated with a logical network within the local area network 100.Similarly, upon being powered or switched off or otherwise beingdisconnected from the network 100, the status of the network device 102would be known and stored by a cache (not shown) associated with thenetwork 100. For example, cloud network 114 may include storage (e.g.cache) that stores the status of the network devices within each localarea network 100 it is connected to and/or provides access to. Inanother example, the gateway 110 may include storage that stores thestatus of the network devices within each local area network it isconnected to and/or provides access to. More specifically, the statusstored in the cache may include a status table which indicates thecurrent status of each network device (as of its last communication witheach network device). A status table may include all statuses of eachnetwork device, or individual storage tables for each local area networkor other subset of its network devices/networks. In one embodiment, achange in status may prompt the network device to push its change in instatus to the cloud network 114 for storage or updating of the cloud'sstored status table. In another embodiment, cloud network 114 and/orgateway 110 may continuously (or periodically) communicate with eachnetwork device to check to see if its status has changed.

In some embodiments, a network device (e.g. network device 102 and/or104) may, upon connecting to the local area network 100, check thestatus of the network devices on the network 100. In other embodiments,one network device may check the status of one or more of the othernetwork devices on the network 100. The network device may seek to checkthe status of another network device or access device for variousreasons, including to display such status(es) to a user on a display orotherwise, to check whether that network device belongs to the samenetwork, to synchronize or coordinate any scheduled executions, toupdate an attribute based on adjustment received among others. Forexample, a network device or user may desire to check various statuseson a connected device, such as power level, timestamped activity history(e.g. temperature for a thermostat, motion for a motion detector, etc.),how long it has been active/turned on, attributes for operation of theconnected device (e.g., a brightness of a lamp, a speed of a fan, or asensitivity of a sensor, etc.), among many others.

In some embodiments, a device, such as the access device 108 shown inFIG. 1 or the gateway 110, connected to the network 100 can communicatean updated status of a network device, such as the network devices 102and/or 104. The updated status can be communicated via the network 100and can include an adjustment that affects a status of the networkdevice. The adjustment can include an amount of change to one or moreattributes, one or more settings, or a combination thereof related tooperation of the network device connected to the network 100. The accessdevice 108 or the gateway 110 can present a graphical interface that canreceive input corresponding to an adjustment to a status of a device. Insome embodiments, the updated status of the network device communicatedto the network 100 can be received by a network device to which theupdated status applies, or can be received by the gateway 110, the cloudnetwork 110, or any other device in communication with the network. Ifthe device cannot directly receive the updated status, it can alsoreceive the updated status from the cloud network 114, the gateway 110,or the other devices in the network 100. In some embodiments, thenetwork device can communicate its updated status to the network 100,which can indicate whether the status has been updated. The updatedstatus can be received by the access device or any other device in thenetwork 100. In some embodiments where the access device is not locatedwithin the network 100, the access device may not immediately receivethe updated status. The updated status can be stored by the cloudnetwork 114 or the gateway 110 for communication to the access device.The status of the network device can indicate whether an adjustment wasmade based on an adjustment in a setting or an attribute transmitted bythe access device. Alternatively, or additionally, the access device canreceive, from any other network device connected to the network 100, astatus update indicating whether the adjustment was in fact made at anetwork device.

A network device seeking to check the status of any other device on thenetwork 100 may communicate with the cloud network 114, to which alldevices on the network 100 are connected either directly or indirectly.Since the cloud network 114 and/or the gateway 110 can store an updatedtable/list of the statuses of each of the network devices 102 and 104within the requesting network's local area network, the cloud network114 and/or gateway 110 may communicate such status data to the networkdevices 102 and 104 and the access device. For example, if networkdevices 102 and 104 were to each turn on and communicate their statusesto cloud network 114, cloud network 114 may analyze the status ofnetwork devices 102 and 104 and communicate to network devices 102 and104 that they are each connected to the same local area network 100.

FIG. 3 illustrates an example of a network 300, according to embodimentsof the present invention. Specifically, the network 300 can be awireless local area network enabling an access device to communicatewith network devices to control adjustment of attributes related tooperation of the network devices. Network 300 includes network device302, network device 304, network device 306, and network device 308. Thenetwork 300 also includes access device 108. In other words, the network300 may be substantially similar to the network 100 except that accessdevice 108 has been turned on near the network 300, to which it isassociated, or has entered an area to which the network 300 can reach.

When access device 108 can enter the network 300 as shown in FIG. 3,access device 108 may be authenticated based on the access device'sauthentication with the logical network or may otherwise commencecommunication with cloud network 114. Access device 108 may alsocommunicate notification of its presence or other information directlyto other network devices 302-308 within network 300, as shown in FIG. 3by communication paths 330. As noted, such communication may includevarious communications protocols, such as Universal Plug and Play(UPnP), Bluetooth®, Zigbee®, Ultra-Wideband (UWB), WiFi-Direct, WiFi,Bluetooth® Low Energy (BLE), sound frequencies, and/or the like. Forexample, access device 108 may communicate to all other devices innetwork 300, including network device 302, network device 304, networkdevice 306, and network device 308, information/data regarding itsstatus. Such status data may include the fact that it is present andturned on, or other status data/information. At any time that networkdevices 302, 304, 306 and 308 recognize that access device 108 ispresent at network 300, the network devices may communicate back toaccess device 108. For example, the network devices may send anacknowledgement (e.g., ACK signal) back to access device 108 to confirmthat they received the status data sent by access device 108. Thenetwork devices may also send their own status data to access device108.

While network devices 302-308 and access device 108 may each receivecommunication from other network devices around the network 300,including the status of each of those network devices, network devices302-308 and/or access device 108 may be continuously scanning network300 (including, for example, running discovery algorithms) to determinewhether any devices within the network have moved, turned on/off orotherwise added to or subtracted from the network 300, or have otherwisechanged statuses.

Since network devices 302-308 and access device 108 may each receivecommunication from other devices around network 300, including thestatus of each of those devices, each network device within network 300may know the status of each other network device in the network 300. Forexample, access device 108 or devices 302-308 may not be required tocommunicate with cloud network 114 in order to obtain one or more ofsuch statuses. Since cloud network 114 is an external network and may beremote from network 300, communication between network devices withinthe network 300 and cloud 114 may take more time than communicationbetween two devices within network 300. For example, communicationbetween devices within network 300 may take anywhere from 1 millisecondto 100 milliseconds, while communication between a device within network300 and the cloud network 114 may take anywhere from 50 milliseconds to1 second or more). Furthermore, if a network device is retrievinginformation from cloud 114, the request must travel from the networkdevice to cloud network 114, and then the information must travel backfrom cloud network 114 to the network device. This process may doublethe latency caused by retrieving information with cloud 114. Therefore,devices within the network 300 may choose to send and receive/retrievestatuses directly with other devices within the network 300 instead ofcommunicating such information via cloud network 114. When a networkdevice receives status data from another network device on the device'slocal area network 300, it may store that status data so that it mayretrieve and use that status data at a later time.

FIG. 4 illustrates an example of a network 400, according to embodimentsof the present invention. The local area network 400 may include networkdevice 302, network device 304, network device 306, network device 308,and access device 108. FIG. 4 also illustrates that one or more networkdevices 302-308 and/or access device 108 may include a storage device,such as a cache, for storing data, including data regarding its ownstatus and data regarding statuses received from the other deviceswithin local area network 400. For example, access device 108 may, afterbeing powered up, broadcast/send its status to network device 308 viacommunication 434. Network device 308 may store the status data receivedfrom access device 108 until the next time access device 108 updates itsstatus by sending new/updated status data to network device 308. Cachemay be used for storage within network devices 302-308 and/or accessdevices within the local area network 400 so that each of the devicesmay be able to quickly retrieve the data it needs from storage. Anapplication operating on the access device 108 can access the cache toobtain information to display the visual interface for each networkdevice 302-308 registered within the network 400. Although a cachingdevice may be used to store such data within the network and/or accessdevices within the local area network 400, other types of storage may beused.

The cache can contain a known interface list including interfaceinformation for different, known types of devices. The known list caninclude a record for each network device known by the access device 108to exist on the network 400. When an application is run on the accessdevice 108, the access device 108 can access the known interfaces in thecache to present the display of access device 108. The display canpresent one or more visual interfaces, each corresponding to a networkdevice known to exist on the network 400. Each visual interface can begenerated based on a visual interface module corresponding to eachdevice on the network 400. In an example, the display can include avisual interface (e.g., a module tile) for each device in the network400 having an interface in the known interface list.

The cache can also contain known status information about each networkdevice in the known device list. When the application is run on theaccess device 108, the access device 108 can access the known statusinformation in the cache to present a status display. The access device108 can populate each tile with an indicator representing the respectiveknown status information for each device in the known device list. Thestatus display can include an indicator of one or more attributes, oneor more settings, or a combination thereof related to operation of eachdevice in the network 400. For example, the status display can include aspeed of a fan (e.g., a fan speed of 56 in a range of values between 0and 100) of the network device 302 (e.g., a fan), a value of sensitivityof a sensor (e.g., a value of 34 in a range of values 0-100) for thenetwork device 304 (e.g., a motion sensor), a value of brightness (e.g.,65 percent brightness) for the network device 306 (e.g., a light bulb),and a value of temperature (e.g. a slow cooker). Although shown ashaving a single indicator for an attribute or a setting related tooperation of a network device, the status display can present aplurality of indicators corresponding to different attributes and/orsettings related to operation of a network device.

In some embodiments, the cache can include other information about anetwork device. The other information can indicate a device's firmwareversion, last known firmware update status, connectivity to cloudstatus, registration status (e.g., whether the network device has a keyor not), and other such information. The cache can include informationthat could be used for troubleshooting. In embodiments described below,the access device 108 can access status information from another otherdevice on the network 400 and can use that information to update its owncache, update the status display, and/or pass the information to thecloud network 114 and/or the gateway 110 for trouble shooting and/orstorage.

Even though each network device may know and store (e.g. in cache) thestate of each other network device within local area network 400, anetwork device may not know when another network device changes status(e.g. turns/powers off). However, network devices and/or access deviceswithin local area network 400 may broadcast/send any updates in itsstatus to other devices on the network. For example, if network device302 changes status, it may send status data to the other networkdevices, such as network devices 304, 306 and 308 and to access device108. However, network device 302 may not know which devices to updatesince the other devices may change statuses periodically (e.g. turnoff).

Therefore, a network or access device may subscribe to another networkor access device within local area network 400. For example, networkdevices 304, 306 and 308 and access device 108 may subscribe to statusdata notifications/updates from network device 302. Such a subscriptionmay be registered for upon initial connection with network device 302when network device 302 first enters local area network 400 or at anyother time after network device 302 has been associated with local areanetwork 400. Subscriptions may be controlled to last indefinitely or mayexpire after a certain predetermined period of time after initialsubscription. However, network devices may re-subscribe to anothernetwork device before or after their previous subscription has expired.

Subscriptions between network device and/or access devices may beregistered, similar to registering a network device upon initialentrance into the local area network, including security registrationsdescribed herein with respect to FIGS. 1 and 2. For example, a networkdevice may send its unique security key, which it may have stored alongwith its network ID after being registered on the network, to a networkdevice to which it wants to subscribe. However, subscriptions may takeon many other forms, including sending a different form ofidentification to a network device to which a network device wants tosubscribe. However, subscriptions may take on many other forms,including sending a different form of identification to a network deviceto which a network device wants to subscribe.

Upon receiving a subscription from another network device or accessdevice, the device being subscribed to may store a list of the devicesthat subscribed to it. For example, network device 302 may store a listof network devices 304, 306 and 308 and access device 108 after thosedevices subscribe to network device 302. Then, when network device 302undergoes a change in status, network device 302 may send that change instatus to only the devices that had previously subscribed to it butwhere the subscription had not yet expired. Furthermore, according tosome embodiments, the subscription list of a network device may beautomatically updated if that device receives notification that anotherdevice has left the range of the local area network, either from thatdevice itself or from a different device. Therefore, the various deviceswithin a given local area network, such as network 400, each containcontinuously updated statuses of each other device on the network andobtain those statuses and updates through direct communication withoutnecessary use of the cloud.

FIG. 5 illustrates an access device 108 that is located remotely fromnetwork 500 (e.g. local area network), according to embodiments of thepresent invention. Local area network 500 includes gateway 110 andnetwork devices 502 and 504 (which may be, for example, the same as anyof network devices 302-308 in FIGS. 3 and 4), as shown in FIG. 5.However, network 500 may also include a variety of other network devicesand one or more access devices directly connected to network 500.Gateway 110 is connected to cloud network 114, and allows networkdevices 502 and 504 to connect to cloud 114, the internet, or otherexternal networks via gateway 110. In some embodiments, the networkdevices 502 and 504 may include home automation devices that allow auser to access, control, and/or configure various home applianceslocated within the user's home, such as a television, radio, light,microwave, iron, and/or the like.

Access device 108 is not directly connected to network 500. Instead,access device 108 is external to network 500 and may connect to cloudnetwork 114 and to network 500 via cloud network 114. As noted, networkdevices 502 and 504 may change status on a periodic basis. In someembodiments, even when external to and not directly connected to network500, an access device may request to check the status of the devices onthe network. When access device 108 seeks to check the status of anydevice on the network, the access device 108 may transmit/send acommunication 536 to the cloud network 114, to which all devices on thenetwork are connected either directly or indirectly via gateway 110.Since the cloud network 114 stores an updated table/list of the statusesof each of the devices within the requesting access device's network,the cloud network 114 may transmit a communication 538 of such statusdata to the access device 108. For example, after network devices 502and 504 are turned on, authenticated and are a part of network 500,network devices 502 and 504 may communicate their statuses to cloudnetwork 114. Furthermore, any time the status of network devices 502 and504 changes, the device that incurred a status change may push/sendinformation (e.g. an indication) of that status change to cloud network114. Cloud network 114 may store, in cache 526 or otherwise, thestatuses (which may be time stamped in metadata or otherwise) of networkdevices 502 and 504. Therefore, when access device 108 requests fromcloud network 114 the statuses of devices on network 500, cloud 114 maysend its most recently stored/updated statuses to access device 108.

To obtain the most updated status data of devices within network 500,cloud 114 may, upon receiving a request for status data related tonetwork devices 502 and 504, transmit/send a communication 532 (e.g.request, query, etc.) for such status data to network devices 502 and504 via gateway 110. Once network devices 502 and 504 receive thisrequest, network devices 502 and 504 may send a communication 534 (e.g.updated status data) to cloud 114 to replace the previouslystored/cached statuses in cache 526. Upon receipt of updated status data534 from network 500, cloud 114 may send a communication 538 of suchstatus data to the access device 108.

However, the process of cloud network 114 requesting updated statusesfrom network devices 502 and 504 within network 500 may cause latencywithin the system. More specifically, the time required for cloudnetwork 114 to request updated statuses from network devices 502 and 504and to in turn receive updated statuses from network devices 502 and 504may be substantially greater than the time required for cloud network114 to send its currently stored statuses (without being updated) fornetwork devices 502 and 504 to access device 108. For example, of thetotal time required for access device 108 to receive updated statusesfrom cloud network 114, 80% or more of that total time may include cloudnetwork 114 requesting updated statuses from network devices 502 and504. On the other hand, of the total time required for access device 108to receive updated statuses from cloud network 114, 20% or more of thattotal time may include the status data being transmitted from cloudnetwork 114 to access device 108. Since a majority of the processrequired for access device 108 to request and receive status data fornetwork devices 502 and 504 is the transmission of data between cloud114 and network devices 502 and 504, the access device 108 and cloudnetwork 114 may maximize efficiency by minimizing the effect of thetransmission of data between cloud 114 and network devices 502 and 504on the whole process/system.

FIG. 6 illustrates an example of a front view of a network device 600.FIG. 7 illustrates an example of a side view of the network device 600.The network device 600 may include any of the network devices 102, 104,or 106 described herein. In some embodiments, the network device 600 maybe a home automation network device. For example, the network device 600may include a home automation switch that may be coupled with a homeappliance. A user may wirelessly access the network device 600 in orderto access, control, and/or configure various home appliances locatedwithin the user's home. For instance, the user may remotely controlappliances such as a television, radio, light, microwave, iron, spaceheater, wall A/C unit, washer, dryer, fan, and/or the like.

In some embodiments, the network device 600 may include a WiFi enabledswitch that connects home appliances and other electronic devices to acompatible 802.11b/g/n/ac WiFi network. The network device 600 may thusallow users to locally or remotely turn devices on or off from anywhere,program customized notifications, and/or change device status. Thenetwork device 600 may further allow a user to create custom schedulesor have devices respond to sunrise or sunset.

The network device 600 includes an power switch 602 that may bedepressed in order to turn the network device 600 on and off. In someembodiments, a light source may be integrated with or located behind thepower switch. For example, a light-emitting diode (LED) may be locatedon a circuit board under the power button 602. The light source may beilluminated when the network device 600 is powered on, and may not beilluminated when the network device 600 is powered off.

The network device 600 further includes a communications signalindicator 604. The signal indicator 604 may indicate whether the networkdevice 600 has access to a communications signal, such as a WiFi signal.For example, the signal indicator 604 may include a light source (e.g.,a LED) that illuminates when the network device 600 is connected to acommunications signal. The light source may depict different colors orother characteristics (e.g., flashing, dimming, or the like) to indicatedifferent levels of signal strength or mode of operation.

The network device 600 includes a restore button 710. The restore button710 may allow a user to reset the network device 600 to factory defaultsettings. For example, upon being depressed, the restore button 710 maycause all software on the device to be reset to the settings that thenetwork device 600 included when purchased from the manufacturer.

The network device 600 further includes a plug 708 and an outlet 606.The plug 708 allows the network device 600 to be plugged into a wallsocket, such as a socket providing 120V, 220V, or the like. In turn, anappliance may be plugged into the outlet 606. Once the network device600 is registered according to the techniques described above, anappliance plugged into the socket 606 may be controlled by a user usingan access device (e.g., access device 108).

FIG. 8 is an example of a block diagram of the network device 600depicting different hardware and/or software components of the networkdevice 600. As described above with respect to FIGS. 6 and 7, thenetwork device 600 includes the outlet 606, the plug 708, the powerbutton 602, the restore button 710, and the communications signalindicator 604. The network device 600 also includes light source 828associated with the power button 602. As previously described, the lightsource 828 may be illuminated when the network device 600 is powered on.

The network device 600 further includes a relay 810. The relay 810 is aswitch that controls whether power is relayed from the plug 708 to theoutlet 606. The relay 810 may be controlled either manually using thepower button 602 or remotely using wireless communication signals. Forexample, when the power button 602 is in an ON position, the relay 810may be closed so that power is relayed from the plug 708 to the outlet606. When the power button 602 is in an OFF position, the relay 810 maybe opened so that current is unable to flow from the plug 708 to theoutlet 606. As another example, an application or program running on anaccess device may transmit a signal that causes the relay 810 to beopened or closed. For instance, an access application may display agraphical interface on the access device that includes a power button.The user may tap or otherwise select the power button, and the accessapplication may send a communication signal (e.g., over a WiFi network)to the network device 600 instructing the network device 600 to open orclose the relay 810.

The network device 600 further includes flash memory 820 and dynamicrandom access memory (DRAM) 822. The flash memory 820 may be used tostore instructions or code relating to an operating system, one or moreapplications, and any firmware. The flash memory 820 may includenonvolatile memory so that any firmware or other program can be canupdated. In the event the network device 600 loses power, informationstored in the flash memory 820 may be retained. The DRAM 822 may storevarious other types of information needed to run the network device 600,such as all runtime instructions or code.

The network device 600 further includes a CPU/Radio 818. The CPU/Radio818 controls the operations of the network device 600. For example, theCPU/Radio 818 may execute various applications or programs stored in theflash memory 820 and/or the dynamic random access memory (DRAM) 822. TheCPU/Radio 818 may also receive input from the various hardware andsoftware components, interpret the input, and perform one or morefunctions in response to the input. As one example, the CPU/Radio 818may determine whether the power button 602 has been pressed, anddetermines whether the relay 810 needs to be opened or closed. TheCPU/Radio 818 may further perform all communications functions in orderto allow the network device 600 to communicate with other networkdevices, one or more gateways, a cloud network, and/or one or moreaccess devices. While the CPU and radio of the network device 600 areshown to be combined in the CPU/Radio 818, one of ordinary skill in theart will appreciate that, in some embodiments, the CPU and radio may beseparately located within the network device 600. For example, CPUcircuitry may be situated at a separate location on a circuit board fromthe location of radio circuitry, the CPU circuitry may be located on adifferent circuit board from the radio circuitry, or the like. Further,the network device 600 may include multiple radios that are configuredto communicate using one or more communication protocols, such as anycombination of a WiFi™ transceiver radio, a Bluetooth™ transceiverradio, a Zigbee™ transceiver radio, a UWB transceiver radio, aWiFi-Direct transceiver radio, a BLE transceiver radio, and/or any otherwireless network transceiver radio or interface. In some embodiments,the network device 600 does not include a cellular network transceiverradio or interface, and thus may not be configured to directlycommunicate with a cellular network. In some embodiments, the networkdevice 600 may include a cellular network transceiver radio, and may beconfigured to communicate with a cellular network using the cellularnetwork transceiver radio.

The network device 600 may communicate with other devices and/ornetworks via antenna 824. For example, antenna 824 may include a 2.4 GHzantenna, a 5 GHz antenna, or the like, that can transmit and receiveWiFi communications signals. The network device 600 may include othertypes of antennas that can communicate Bluetooth® signals, Zigbee®signals, Ultra-Wideband (UWB) signals, WiFi-Direct signals, BLE signals,and/or the like. In some embodiments, the antenna 824 may be configuredto communicate different types of signals, such as the WiFi signals,Bluetooth® signals, Zigbee® signals, UWB signals, WiFi-Direct signals,BLE signals, and/or the like. In some embodiments, the network device600 may include multiple antennas for communicating the different typesof communication signals. As one example, the network device 600 mayinclude both a 2.4 GHz antenna and a 5 GHz antenna.

The network device 600 further includes a driver 816, a switching powersupply 812, and a voltage regulator 814. The driver 816 may includeinstructions or code that can be used to translate control signals orcommands received from applications running on the DRAM 822 to commandsthat the various hardware components in the network device 600 canunderstand. In some embodiments, the driver 816 may include an ambientapplication running on the DRAM 822. The switching power supply 812 maybe used to transfer power from the outlet in which the plug 708 isconnected to the various loads of the network device 600 (e.g.,CPU/Radio 818). The switching power supply 812 may efficiently convertthe voltage and current characteristics of the electrical power to alevel that is appropriate for the components of the network device 600.For example, the switching power supply 812 may perform AC-DCconversion. In some embodiments, the switching power supply 812 may beused to control the power that is relayed from the plug 708 to theoutlet 606. The voltage regulator 814 may be used to convert the voltageoutput from the switching power supply 812 to a lower voltage usable bythe CPU/Radio 818. For example, the voltage regulator 814 may regulatethe DC voltage from 5V to 3.3V.

In various embodiments, functions may be stored as one or morecomputer-program products, such as instructions or code, in anon-transitory machine-readable storage medium, such as the flash memory820 and/or the DRAM 822. The network device 600 can also comprisesoftware elements (e.g., located within the memory), including, forexample, an operating system, device drivers, executable libraries,and/or other code, such as one or more application programs, which maycomprise computer programs implementing the functions provided byvarious embodiments, and/or may be designed to implement methods and/orconfigure systems, as described herein. Merely by way of example, one ormore procedures described with respect to the processes discussed above,for example as described with respect to FIG. 2, may be implemented ascode and/or instructions executable by a computer (and/or a processorwithin a computer); in an aspect, then, such code and/or instructionscan be used to configure and/or adapt a computer (or other device) toperform one or more operations in accordance with the described methods.Such functions or code may include code to perform the steps describedabove with respect to FIG. 2. The memory, such as the flash memory 820and/or the DRAM 822, may be a processor-readable memory and/or acomputer-readable memory that stores software code (programming code,instructions, etc.) configured to cause a processor(s) within theCPU/Radio 818 to perform the functions described. In other embodiments,one or more of the functions described may be performed in hardware.

A set of these instructions and/or code might be stored on anon-transitory machine-readable storage medium, such as the flash memory820 and/or the DRAM 822. In some cases, the storage medium might beincorporated within a computer system, such as the CPU/Radio 818. Inother embodiments, the storage medium might be separate from a computersystem (e.g., a removable medium, such as a compact disc), and/orprovided in an installation package, such that the storage medium can beused to program, configure and/or adapt a computer with theinstructions/code stored thereon. These instructions might take the formof executable code, which is executable by the network device 600 and/ormight take the form of source and/or installable code, which, uponcompilation and/or installation on the network device 600 (e.g., usingcompilers, installation programs, compression/decompression utilities,etc.) then takes the form of executable code.

It should be appreciated that the network device 600 may have othercomponents than those depicted in FIGS. 6-8. Further, the embodimentshown in the figures are only one example of a network device that mayincorporate an embodiment of the invention. In some other embodiments,network device 600 may have more or fewer components than shown in thefigure, may combine two or more components, or may have a differentconfiguration or arrangement of components.

FIG. 9 is a schematic illustration of a local area network 900 includinga network device 902 that includes an appliance 950. The network device902 can comprise an interface device 904 and the appliance 950 connectedby an appliance interface 908. The appliance interface 908 can include adata connection 918 and a power connection 916. The data connection 918can be a serial connection (e.g., RS-232, USB, or other), or any othersuitable data connection. The interface device 904 can be fully poweredby the appliance 902 through the power connection 916, or can have aseparate source of power.

The appliance 950 can be any suitable electric device, such as a crockpot, space heater, an iron, a washing machine, a dishwasher, a lamp, aradio, a computer, an amplifier, or another electrical device.Additional examples of suitable electrical devices include electricaldevices incorporated into or with non-electrical devices, such as anactuator system in an electrically-actuated deadbolt, a sensing systemin a seat cushion, or other suitable electrical device incorporated intoor with a non-electrical device. The appliance 950 can be adapted tooperate with the interface device 904. The appliance 950 can be anyfinite state machine. The appliance 950 can, but need not, know or storeone or more states related to the appliance. For example, the appliance950 may know or store data related to whether the appliance 950 isturned on, how long the appliance has been on (or off), among otherstatus data.

The interface device 904 can be positioned within the housing of theappliance 950, or can be attached externally to the appliance 950. Theinterface device 904 can be removable from the appliance 950, or can bepermanently installed in or on the appliance 950.

The interface device 904 can be connected to the local area network 900through a network interface. The interface device 904 can be connectedby a wired or wireless connection (e.g., WiFi, Zigbee, or othersdescribed herein or well known). In some embodiments, the interfacedevice 904 can be connected directly to the cloud network 114 through acellular internet connection (e.g., EDGE, LTE, or others).

The interface device 904 can communicate with another network device, anaccess device 108, or another client device through the networkinterface 906. The interface device 904 can transmit a statusinformation signal 910 with status information to the access device 108,and the access device 108 can transmit a network device control signal912 to the interface device 904. The status information signal 910 andthe network device control signal 912 can be transmitted between theinterface device 904 and the access device 108 using atelecommunications network (e.g., a cellular network, or other suitablebroadband network), using a local area network 900 (e.g., through agateway 110), or using the cloud network 114, although such a signal maypass through an intermediary device or network to do so.

The interface device 904 can interpret the network device control signal912 and perform actions based on the contents of the network devicecontrol signal 912. The network device control signal 912 can includecommands that can be performed by the interface device 904 itself. Thenetwork device control signal 912 can also include commands that are tobe performed by the appliance 950. Commands that are to be performed bythe appliance 950 can include commands like turn on or off, set adesired temperature (e.g., heat up or cool down to 215° F. or any othertemperature), or other suitable commands depending on the particularappliance. The interface device 904 can interpret the network devicecontrol signal 912 and can send out a command 922, through the dataconnection 918 of the appliance interface 908, based on the networkdevice control signal 912. The appliance 950 can then perform thecommand indicated in the network device control signal 912.

The interface device 904 can also transmit commands to the appliance 950that are not based on a network device control signal received from theaccess device 108, but are rather based on programming in the interfacedevice 904. Examples of such commands can include commands to update acommunication rate, commands to check a state of the appliance 950,commands to set or get a clock time of the appliance 950, or any othersuitable commands.

The interface device 904 can receive, through the data connection 918 ofthe appliance interface 908, a response (e.g., response 920) to anycommand from the appliance 950. In some examples, the response 920 caninclude an indication that the command 922 was received. In someexamples, the response may include only an indication that a command isreceived (e.g., an ACK). In some examples, the response 920 can includeinformation for some value on the appliance 950, such as an “on/off”state, a serial number, a product identification, a manufactureridentification, a temperature, a time since live, a setting, or anyother value retrievable from the appliance 950. The interface device 904can interpret the value and can send information about the value (e.g.,the state of the appliance is “on,” the temperature of the appliance,the time since the appliance first turned on, or other information) asstatus information (e.g. using status information signal 910) to theaccess device 108. Additionally, the interface device 904 can sendstatus information about itself (e.g., time since live, supplied power,signal strength, and others) as status information (e.g. using statusinformation signal 910) to the access device 108.

The interface device 904 can also use responses (e.g., response 920)from the appliance 950 to perform additional functions at the interfacedevice 904, such as error handling. In some cases, when performing theadditional functions, the interface device 904 does not transmit anystatus information 910 to the access device 108 based on thoseparticular responses.

The access device 108 can include one or more display tiles (e.g.,display tile 914) for displaying information and controls correspondingto the network device 102.

In some embodiments, the interface device 904 can transmit a heartbeatcommand (e.g., command 922) over the data connection 918 to theappliance 902 to determine whether the appliance 950 is working properlyand/or in a state of readiness. If the interface device 904 determinesthat the appliance 950 has had some sort of failure (e.g., the appliance950 sends a response 920 indicating a failure or the interface device904 does not receive any response 920), the interface device 904 cantake corrective action (e.g., restarting the appliance 950 or an elementof the appliance 950), can log the event, or can alert the user).

FIG. 10 depicts a block diagram of a network device including aninterface device 904 attached to an appliance 950 according to oneembodiment. The interface device 904 can include connector 1012 thatinteracts with connector 1032 of the appliance 950.

The interface device 904 can include flash memory 1004 and dynamicrandom access memory (DRAM) 1006. The flash memory 1004 may be used tostore instructions or code relating to an operating system, one or moreapplications, and any firmware. The flash memory 1004 can be used tostore a cache. The flash memory 1004 may include nonvolatile memory sothat any firmware or other program can be can updated. In the event theinterface device 904 loses power, information stored in the flash memory1004 may be retained. The DRAM 1006 may store various other types ofinformation needed to run the interface device 904, such as all runtimeinstructions or code. The flash memory 1004 or DRAM 1006 or acombination thereof may include all instructions necessary tocommunicate with an appliance 950, including all instructions necessaryto communicate using the appliance serial protocol disclosed herein.

The interface device 904 further includes a CPU/Radio 1002. TheCPU/Radio 1002 can control the operations of the interface device 904.For example, the CPU/Radio 1002 may execute various applications orprograms stored in the flash memory 1004 and/or the dynamic randomaccess memory (DRAM) 1006. The CPU/Radio 1002 may also receive inputfrom the appliance 950, interpret the input, and perform one or morefunctions in response to the input. The CPU/Radio 1002 may furtherperform all communications functions in order to allow the interfacedevice 904 to communicate with other network devices, one or moregateways, a cloud network, and/or one or more access devices. Theinterface device 904 may communicate with other devices and/or networksvia antenna 1026. For example, antenna 1026 may include a 2.4 GHzantenna that can transmit and receive WiFi communications signals 1028.The antenna 1026 may include other types of antennas that cancommunicate Bluetooth® signals, Zigbee® signals, Ultra-Wideband (UWB)signals, and/or the like. In some embodiments, the interface device 904may include multiple antennas for communicating different types ofcommunication signals.

The CPU/Radio 1002 can include at least one universal asynchronousreceiver/transmitter (UART) 1010. The CPU/Radio 903 can use the UART1010 to send and receive serial communications. The CPU/Radio 903 cansend data through a transmit line 1022 and a receive data through areceive line 1024. The CPU/Radio 903 can send and receive data throughthe transmit line 1022 and receive line 1024 using a serial protocol,such as RS232. The CPU/Radio 1002 can also include an input/output(GPIO) line 1014, a restore line 1016, an LED 1 line 1018, and an LED 2line 1020. The CPU/Radio 1002 can have additional or fewer lines asnecessary. The GPIO line 1014 can be used for any suitable function,such as powering an indicator light on an appliance 950 or accepting aninput from the appliance 950. A signal sent on the restore line 1016 canbe used to restore the CPU/Radio 1002 and/or the interface device 904 tofactory defaults. The LED 1 line 1018 and LED 2 line 1020 can be used topower first and second LEDs that can be used to indicate variousstatuses, such as whether the interface device has a network connectionand whether the interface device is powered on.

The interface device 904 further includes a voltage regulator 1008. Thevoltage regulator 1008 may be used to convert the voltage output fromthe appliance 950 to a voltage usable by the CPU/Radio 1002. Forexample, the voltage regulator 1008 may regulate the DC voltage from 5Vto 3.3V. The voltage regulator 1008 can be supplied with power from apower line 1030.

Each of the interface lines, including the GPIO line 1014, the restoreline 1016, the LED 1 line 1018, the LED 2 line 1020, the transmit line1022, the receive line 1024, the power line 1030, and any additionallines, can be routed through connector 1012. Connector 1012 can be aproprietary or universal connector. Any appliance 950 to which theinterface device 904 is attached through the connector 1012 can have thenecessary hardware to make use of the interface lines, such as toprovide power to the power line 1030 and to provide the first and secondLEDs that are driven by the LED 1 line 1018 and LED 2 line 1020.

In alternate embodiments, some interface lines are not routed throughthe connector 1012. For example, the power line 1030 can be routed to apower supply attached directly to the interface device 904, and the LED1 line 1018 and LED 2 line 1020 can be routed to first and second LEDslocated within the interface device 904.

In various embodiments, functions may be stored as one or moreinstructions or code in memory, such as the flash memory 1004 and/or theDRAM 1006. The interface device 904 can also comprise software elements(e.g., located within the memory), including, for example, an operatingsystem, device drivers, executable libraries, and/or other code, such asone or more application programs, which may comprise computer programsimplementing the functions provided by various embodiments, and/or maybe designed to implement methods and/or configure systems, as describedherein. Merely by way of example, one or more procedures described withrespect to the processes discussed below may be implemented as codeand/or instructions executable by a computer (and/or a processor withina computer); in an aspect, then, such code and/or instructions can beused to configure and/or adapt a device (e.g. a specialty computer) toperform one or more operations in accordance with the described methods.Such functions or code may include code to perform various stepsdescribed below. The memory, such as the flash memory 1004 and/or theDRAM 1006, may be a processor-readable memory and/or a computer-readablememory that stores software code (programming code, instructions, etc.)configured to cause a processor(s) within the CPU/Radio 1002 to performthe functions described. In other embodiments, one or more of thefunctions described may be performed in hardware.

A set of these instructions and/or code might be stored on acomputer-readable storage medium, such as the flash memory 1004 and/orthe DRAM 1006. In some cases, the storage medium might be incorporatedwithin a computer system, such as the CPU/Radio 1002. In otherembodiments, the storage medium might be separate from a computer system(e.g., a removable medium, such as a compact disc), and/or provided inan installation package, such that the storage medium can be used toprogram, configure and/or adapt a device (e.g. a computer) with theinstructions/code stored thereon. These instructions might take the formof executable code, which is executable by the interface device 904and/or might take the form of source and/or installable code, which,upon compilation and/or installation on the interface device 904 (e.g.,using any of a variety of compilers, installation programs,compression/decompression utilities, etc.) then takes the form ofexecutable code.

Substantial variations may be made in accordance with specificrequirements. For example, customized hardware might also be used,and/or particular elements might be implemented in hardware, software(including portable software, such as applets, etc.), or both. Further,connection to other access or computing devices such as networkinput/output devices may be employed.

The interface device 904 may have other components than those depictedin FIG. 10. Further, the embodiment shown in the figures are only oneexample of an interface device that may incorporate an embodiment of theinvention. In some other embodiments, interface device 904 may have moreor fewer components than shown in the figure, may combine two or morecomponents, or may have a different configuration or arrangement ofcomponents.

The appliance 950 can have a processor 1034. The processor 1034 can be amicrocontroller, such as a Peripheral Interface Controller (PIC). Theappliance 950 can include a memory 1036 (e.g., a flash memory or other)that is readable by the processor 1034. The memory 1036 can includeinstructions enabling the innate functionality of the appliance 950,such as heating and timing for a crock pot.

The appliance 950 can include a user interface 1038. The user interface1038 can provide buttons, displays, LEDs, knobs, and other input andoutput elements necessary for a user to interact with the appliance 950.For example, a user interface 1038 for a slow cooker can include adisplay, a power button, a temperature adjustment button, and a startbutton. The user interface 1038 can be driven and/or monitored by theprocessor 1034. In some embodiments, the appliance 950 is “headless” orhas no user interface 1038.

The appliance 950 can include a power supply 1040 that can provide powerto the voltage regulator 1038 of the interface device 904 throughconnector 1032, connector 1012, and power line 1030.

The appliance 950 can include an interface device user interfaceextension 1042. The interface device user interface extension 1042 caninclude various input and output elements that are passed directly tothe interface device 904 without being processed by the processor 1034.Examples of input and output elements of the interface device userinterface extension 1042 include LEDs associated with the LED 1 line1018 and LED 2 line 1020, a hardware restore button associated with therestore line 1016, or any other suitable input/output element.

A local area network, such as a user's home local area network, caninclude multiple network devices that provide various functionalities.Network devices may be accessed and controlled using an access deviceand/or one or more network gateways. One or more gateways in the localarea network may be designated as a primary gateway that provides thelocal area network with access to an external network. The local areanetwork can also extend outside of the user's home and may includenetwork devices located outside of the user's home. For instance, thelocal area network can include network devices such as exterior motionsensors, exterior lighting (e.g., porch lights, walkway lights, securitylights, or the like), garage door openers, sprinkler systems, or othernetwork devices that are exterior to the user's home. It is desirablefor a user to be able to access the network devices while located withinthe local area network and also while located remotely from the localarea network. For example, a user may access the network devices usingan access device within the local area network or remotely from thelocal area network. As explained herein, techniques are provided thatdynamically register and distribute resource bundles associated withnetwork devices, thus allowing network devices to be controlled in asingle application using interfaces defined by resource bundles. Thesetechniques allow a user, whether located locally or remotely from thelocal area network, to access, monitor, and control all network devicesin the local area network.

In an embodiment, a network device can be discovered by an accessdevice, which then compares a unique identifier of the network devicewith a list of known resource bundles. The resource bundles can definerespective interface modules for network devices. If the access devicedoes not find a resource bundle for the network device in a local cacheat the access device, the access device can display a generic interfacefor the type of network device it is while searching a cloud basedservice for an updated, more specific interface module for the networkdevice.

According to embodiments, when a network device is initially connectedto a network, it and an access device can exchange communications toprovide the access device with information that can be used to determinea basic, default visual interface for the network device. Thecommunications can convey the fundamental types of interactions thenetwork device supports. For example, complex interactions between theaccess device and the network device can be simplified and distilleddown to a relatively small set of classifications of interactions alongwith a certain limited set of characterizing attributes associated witheach classification. In one non-limiting embodiment, there are sevenclassifications of interactions in the set of classifications. Oneexample set of classifications includes the following interactionclassifications: 1) Controllable or Observable; 2) Discrete orContinuous; 3) Bound or Unbound (e.g., bounded or unbounded); 4) Restfulor Restless; 5) Fixed or Open State Transition Path; 6) Cyclical orLinear; and 7) Temporal (e.g., repetitive or non-repetitive over time).Examples of these interaction classifications are provided in thefollowing paragraphs. According to embodiments, there is a fundamentalminimum set of attributes that accompany each of these exemplary classesor types of interactions. For example, by using an associated set ofattributes for each interaction type supported by a network device, acomplete and fundamental model describing all interactions for thenetwork device can be formed. This fundamental model can be used todetermine a basic, default visual interface for the network device. Forinstance, such a model can be used to create a default visual interfacein cases where a resource bundle for the network device is unavailable(e.g., when the bundle has not been developed and/or cloud connectivityis unavailable).

In some embodiments, controllable network devices are network devicesthat a user or an access device can interact with and manipulate in somephysical way. For example, a default visual interface module can beconfigured to display one or more interface elements on a display of anaccess device, where the interface elements are usable to control thenetwork device. The visual interface may not map out every physicallypossible activity or interaction with the network device. For example,picking up a network device and moving it to another physical locationmay not be within the scope of interactions and controls supported by avisual interface. Instead, a default visual interface module for thenetwork device can define the supported interactions between the accessdevice and the network device and attempt to capture the fundamentalways in which the access device can interact with the network devicewhen performing basic functions.

According to some examples, observable interactions are provided bynetwork devices that are configured to detect changes in the physicalworld in some way. For example, sensors that are able to monitor for,detect, measure, and convey changes in the observable environmentsupport observable interactions. Non-limiting examples of such networkdevices include motion sensors, moisture sensors, thermometers,thermostats, microphones, pressure sensors, gas/smoke detectors, and thelike. Such sensors can be configured to detect motion, temperature,sound, a change in air pressure, presence of a gas, presence of smoke,presence of a device, presence of a person, or presence of a liquid. Inan embodiment, a change in the environment is observable if it can benoticed or perceived, and registered as being a significant change.

In some embodiments, discrete interactions define a fixed, finite limitto the number of states that a controllable thing (e.g., a networkdevice) has. For example, discrete interactions can also define a finitenumber of states that an observable thing can be detected, noticed, orperceived to be in. Generally, digital or stepped mechanical devices arediscrete. For example, a light switch network device and a digitaltelevision tuner support a discrete number of interactions. In contrast,continuous interactions apply to devices that have no finite number ofcontrollable or observable states. Such devices can a virtually anunlimited number of continuous states. For example, analog continuousmotion mechanical devices can be classified as continuous. Suchcontinuous devices can include, but are not limited to, an analogthermostat, an analog joystick, a lever, a vehicle's accelerator pedal,a mechanical dial, a water faucet, and the like. Continuous devices canalso be inferred to support linear controls (i.e., have linearity). Forexample, a temperature setting or range is continuous and linear. Thatis, an analog thermostat or thermometer cannot go from or detect atemperature of 75 degrees and instantly jump to 80 degrees withoutincreasing continuously from 75 up to 80 degrees.

According to embodiments, device interactions are classified as boundedif they have a lower and/or upper bound. A given network device may havecertain functionalities or controls that are bounded, while also havingother functionalities or controls that are bounded. That is, a devicecan be partially bound, fully bounded, or unbounded. For example, atypical gate or door has a fully open state (i.e., maximum degree ofopenness) and fully closed state (i.e., minimum degree of closed). Sucha gate or door can be classified as fully bounded. A steering wheel, arotary dial, or an analog knob (e.g., a potentiometer with a sliding orrotating contact) are other examples of fully bounded controllabledevices. An escalator, a revolving door, and a treadmill are examples ofpartially bound devices. This is because such devices generally continuein one direction without a bound, but they are bound at some point frommoving in another direction. A bicycle wheel, an electric motor, aturntable, and a rotating tray (e.g., a lazy Susan) are examples ofunbounded devices. Such unbounded devices can spin or rotate withoutbound in either direction. In certain embodiments, a majority ofcontrollable or observable devices can be classified as being bounded.

Restful devices are devices that default back to a known, ‘restful’state when external forces cease to be applied to them. That is, restfuldevices spring back or revert to their restful state when an externalforce is no longer applied to them. Doorknobs, accelerator pedals, andreset buttons are examples of restful devices. Non-restful devicesmaintain their most recent state when external forces cease to beapplied. Light switches, dead bolts, water faucets, and thermostats, areexamples of non-restful devices.

Fixed state devices are devices that can be locked into state transitionpaths that must be followed. Analog tuners, dishwashers with set cycles,and traffic signals for intersections are examples of fixed statedevices. Open state transition devices are not locked into anyparticular state transition path and can take many different paths totransition from one state to any other state. Digital television tunersand keyboards are examples of open state transition devices.

Cyclic devices are devices that cycle back from an upper bound back totheir lower bound and/or vice versa. A combination lock is an example ofa cyclic controllable device. Linear devices do not circularly connectfrom one end to the other. Most devices can be categorized as eitherbeing linear or cyclic.

Temporal devices derive or represent their state over some period oftime rather than a single point in time. That is, temporal devices mustbe observed over some period of time before rendering an observation. Aseat belt warning tone generator, blinking hazard lights, anambulance/emergency siren are examples of temporal devices. Temporaldevices can have a direction and speed of travel of a state. Forexample, a network device embodied as a garage door opener can representthe state of a garage door as closing or opening. Also, for example, aspace heater network device can represent the state of a room where itis located as heating. Similarly, a wall or window air conditioner (A/C)network device can represent the state of a room where it is located ascooling. The observable state of repetitive things can be accuratelydetermined through observation over time as opposed to a single snapshot at a discrete point in time.

Using the above-described exemplary classifications, consider an examplenetwork device that supports interactions via an input device such as ananalog joystick. Such an input device can be defined as beingcontrollable and continuous (i.e., due to its analog mechanics). Theinput device can also be defined as being bounded by the physical limitsof leftmost, uppermost, rightmost, and down most points. Continuing withthe joystick example, this type of input device can be defined as beingrestful at positions x=0 and y=0, with open state transition paths(e.g., it can take any one of many paths to get from the leftmost pointto the rightmost point). The classifications of interactions supportedby a joystick input device can also be defined as being linear andnon-repetitive.

Techniques are described for using third party, run time registerable,discoverable, and auto distributable interfaces for network devices thathave not previously been seen or discovered by an access device in thepast. The techniques dynamically register and distribute resourcebundles defining interfaces for controlling network devices withinwireless and wired networks. In some examples, methods distribute userinterfaces (UIs) for network devices. The network devices can includeinternet of things (IoT) devices. The UIs can be third party, run timeregisterable, discoverable, and auto distributable to an access device.The UIs can be based on network devices that have not previously beenseen by an application on the access device. In some examples, a servermay determine that a resource bundle exists for a network deviceconnected to a home local area network. In one example, the resourcebundle can define a visual interface usable to control a network devicewithin an application executing on an access device.

In other examples, resource bundles can define interfaces for wearabledevices, ambient sensors, and other control interfaces havingdynamically detected and updated interface control capabilities used tocontrol network devices. Such wearable devices or ambient sensors andinterfaces can use interfaces defined by resource bundles, but may nothave visual interfaces. Thus, the interfaces defined by the resourcebundles described herein can be part of visual and non-visualinterfaces. The network device may be a newly discovered network device.The server may provide the resource bundle to an application installedon a mobile device so that the mobile device can use the resource bundleto generate an interface module for monitoring and controlling thenetwork device using the application. The server can also provide theresource bundle to an application installed on a stationary device suchas, for example, a network gateway, a personal computer, a connectedtelevision (i.e., a smart TV), or a touchscreen device mounted on a wallor other surface (e.g., a touchscreen integrated into the door of arefrigerator or another appliance), so that the application can generatea corresponding interface module that is usable to monitor and controlthe network device. For example, a home local area network may include agateway connected to a discovered network device. The server maydetermine that a resource bundle defining an interface module exists forthe network device, and thus that the interface module needs to beprovided to the gateway and/or a mobile device associated with thenetwork device. For example, the server may determine the existence ofmultiple interface modules corresponding to respective network devicesbased on communications from the network devices and/or communicationfrom a mobile device that is associated with the network devices. Upondiscovering a network device that is connected to a home local areanetwork, the server may determine a unique ID for the network device anduse the unique ID to identify an existing interface module for thenetwork device.

In certain embodiments, a network device may have a unique interfaceidentifier assigned to it. For example, a resource bundle for thenetwork device may have been previously uploaded to the server, added toresource bundle registry, and assigned a resource bundle ID. Accordingto these embodiments, the interface defined by such a resource bundle isdesigned for controlling and/or monitoring the network device. Forexample, the interface may be a modular tile for the network device,where the tile is defined by a resource bundle previously created,registered, uniquely identified, and uploaded to a resource bundleregistration and discovery service hosted on the server. The server maythen transmit the uniquely identified resource bundle to an accessdevice associated with the home local area network. Once received by theaccess device, the resource bundle is executable to generate aninterface with interactive elements usable to control the networkdevice. For example, the interface module may be executable within anapplication installed on mobile device and the interface elements mayinclude interactive elements for controlling the network device.Accordingly, an interface module for a newly discovered network devicemay be identified and transmitted to a mobile device so that the mobiledevice can be used to control the network device, regardless of whetherthe mobile device has previously been connected to or associated withthe network device.

In some examples, network devices may send respective communications toa server indicating that they are connected to a local area network. Forexample, a first network device may send a first unique identifier and asecond network device may send a second unique identifier to the server.The server can use the unique identifiers to determine respectiveresource bundles defining interfaces corresponding to each of thenetwork devices. An access device such as a mobile device maysubsequently discover the first and second network devices, retrievetheir unique identifiers, and then use the unique identifiers toretrieve visual interface modules from the server, the interface moduleshaving been designed for controlling the network devices. In an example,the interface modules can be tiles based on standardized designtemplates. For instance, the tiles can be developed in accordance withthe design templates, uniquely identified, and uploaded to the server.In one example, the server can host a cloud-based tile registration anddiscovery service and the tiles can be uploaded to the service by tilecreators and developers, such as network device manufacturers and thirdparty developers.

FIG. 11 illustrates an example of a network 1100, according toembodiments of the present invention. Specifically, the network 1100 canbe a wireless local area network enabling an access device tocommunicate with network devices to control adjustment of attributesrelated to operation of the network devices. Network 1100 includesnetwork device 1102, network device 1104, network device 1106, andnetwork device 1108. In some embodiments, each of the devices 1102,1104, and 1106 may include a home automation network device that allow auser to access, control, and/or configure various appliances, such as atelevision, a space heater, a radio, a light, an electrical switch, afan, a humidifier, a sensor, a microwave oven, an iron, and/or the like.For example, device 1102 may be configured to enable control of a fan,device 1104 may be configured to enable control of a motion sensor,device 1106 may be configured to enable control of a light, and device1108 may be configured to enable control of an electrical switch. Thenetwork 1100 also includes access device 108. In other words, thenetwork 1100 may be substantially similar to the network 100 except thataccess device 108 has been turned on near the network 1100, to which itis associated, or has entered the an area to which the network 1100 canreach.

When access device 108 can enter the network 1100 as shown in FIG. 11,access device 108 may be authenticated based on the access device'sauthentication with the logical network or may otherwise commencecommunication with cloud network 114. Access device 108 may alsocommunicate notification of its presence or other information directlyto other network devices 1102-1108 within network 1100, as shown in FIG.11 by communication paths 1130. As noted, such communication may includevarious communications protocols, such as Universal Plug and Play(UPnP), Bluetooth®, Zigbee®, Ultra-Wideband (UWB), WiFi-Direct, WiFi,Bluetooth® Low Energy (BLE), sound frequencies, and/or the like. Forexample, access device 108 may communicate to all other devices innetwork 1100, including network device 1102, network device 1104,network device 1106, and network device 1108, information/data regardingits status. Such status data may include the fact that it is present andturned on, or other status data/information. At any time that networkdevices 1102, 1104, 1106 and 1108 recognize that access device 108 ispresent at network 1100, the network devices may communicate back toaccess device 108. For example, the network devices may send anacknowledgement (e.g. ACK signal) back to access device 108 to confirmthat they received the status data sent by access device 108. Thenetwork devices may also send their own status data to access device108.

While network devices 1102-1108 and access device 108 may each receivecommunication from other network devices around the network 1100,including the status of each of those network devices, network devices1102-1108 and/or access device 108 may be continuously scanning network1100 (including, for example, running discovery algorithms) to determinewhether any devices within the network have moved, turned on/off orotherwise added to or subtracted from the network 1100, or haveotherwise changed statuses.

Since network devices 1102-1108 and access device 108 may each receivecommunication from other devices around network 1100, including thestatus of each of those devices, each network device within network 1100may know the status of each other network device in the network 1100.For example, access device 108 or devices 1102-1108 may not be requiredto communicate with cloud network 114 in order to obtain one or more ofsuch statuses. Since cloud network 114 is an external network and may beremote from network 1100, communication between network devices withinthe network 1100 and cloud 114 may take more time than communicationbetween two devices within network 1100. For example, communicationbetween devices within network 1100 may take anywhere from 1 millisecondto 100 milliseconds, while communication between a device within network1100 and the cloud network 114 may take anywhere from 50 milliseconds to1 second or more). Furthermore, if a network device is retrievinginformation from cloud 114, the request must travel from the networkdevice to cloud network 114, and then the information must travel backfrom cloud network 114 to the network device. This process may doublethe latency caused by retrieving information with cloud 114. Therefore,devices within the network 1100 may choose to send and receive/retrievestatuses directly with other devices within the network 1100 instead ofcommunicating such information via cloud network 114. When a networkdevice receives status data from another network device on the device'slocal area network 1100, it may store that status data so that it mayretrieve and use that status data at a later time.

FIG. 12 illustrates an example of a network 1200, according toembodiments of the present invention. The local area network 1200 mayinclude network device 1102, network device 1104, network device 1106,network device 1108, and access device 108. FIG. 12 also illustratesthat one or more network devices 1102-1108 and/or access device 108 mayinclude a storage device, such as a cache, for storing data, includingdata regarding its own status and data regarding statuses received fromthe other devices within local area network 1200. For example, accessdevice 108 may, after being powered up, broadcast/send its status tonetwork device 1108 via communication 1234. Network device 1108 maystore the status data received from access device 108 until the nexttime access device 108 updates its status by sending new/updated statusdata to network device 1108. Cache may be used for storage withinnetwork 1200 and/or access devices within the local area network 1200 sothat each of the devices may be able to quickly retrieve the data itneeds from storage. An application operating on the access device 108can access the cache to obtain information to display the visualinterface for each network device 1102-1108 registered within thenetwork 1200. Although a caching device may be used to store such datawithin the network and/or access devices within the local area network1200, other types of storage may be used.

The cache can contain a known interface list including interfaceinformation for different, known types of devices. The known list caninclude a record for each network device known by the access device 108to exist on the network 1200. When an application is run on the accessdevice 108, the access device 108 can access the known interfaces in thecache to present the display of access device 108. The display canpresent one or more visual interfaces, each corresponding to a networkdevice known to exist on the network 1200. Each visual interface can begenerated based on a visual interface module corresponding to eachdevice on the network 1200. In an example, the display can include avisual interface (e.g., a module tile) for each device in the network1200 having an interface in the known interface list.

The cache can also contain known status information about each networkdevice in the known device list. When the application is run on theaccess device 108, the access device 108 can access the known statusinformation in the cache to present a status display. The access device108 can populate each tile with an indicator representing the respectiveknown status information for each device in the known device list. Thestatus display can include an indicator of one or more attributes, oneor more settings, or a combination thereof related to operation of eachdevice in the network 1200. For example, the status display can includea speed of a fan (e.g., a fan speed of 56 in a range of values between 0and 100) of the network device 1102 (e.g., a fan), a value ofsensitivity of a sensor (e.g., a value of 34 in a range of values 0-100)for the network device 1104 (e.g., a motion sensor), a value ofbrightness (e.g., 65 percent brightness) for the network device 1106(e.g., a light bulb), and a value of temperature (e.g. a slow cooker).Although shown as having a single indicator for an attribute or asetting related to operation of a network device, the status display canpresent a plurality of indicators corresponding to different attributesand/or settings related to operation of a network device.

In some embodiments, the cache can include other information about anetwork device. The other information can indicate a device's firmwareversion, last known firmware update status, connectivity to cloudstatus, registration status (e.g., whether the network device has a keyor not), and other such information. The cache can include informationthat could be used for troubleshooting. In embodiments described below,the access device 108 can access status information from another otherdevice on the network 1200 and can use that information to update itsown cache, update the status display, and/or pass the information to thecloud network 114 and/or the gateway 110 for trouble shooting and/orstorage.

Even though each network device may know and store (e.g. in cache) thestate of each other network device within local area network 1200, anetwork device may not know when another network device changes status(e.g. turns/powers off). However, network devices and/or access deviceswithin local area network 1200 may broadcast/send any updates in itsstatus to other devices on the network. For example, if network device1102 changes status, it may send status data to the other networkdevices, such as network devices 1104, 1106 and 1108 and to accessdevice 108. However, network device 1102 may not know which devices toupdate since the other devices may change statuses periodically (e.g.turn off).

Therefore, a network or access device may subscribe to another networkor access device within local area network 1200. For example, networkdevices 1104, 1106 and 1108 and access device 108 may subscribe tostatus data notifications/updates from network device 1102. Such asubscription may be registered for upon initial connection with networkdevice 1102 when network device 1102 first enters local area network1200 or at any other time after network device 1102 has been associatedwith local area network 1200. Subscriptions may be controlled to lastindefinitely or may expire after a certain predetermined period of timeafter initial subscription. However, network devices may re-subscribe toanother network device before or after their previous subscription hasexpired.

Subscriptions between network device and/or access devices may beregistered, similar to registering a network device upon initialentrance into the local area network, including security registrationsdescribed herein with respect to FIGS. 1 and 2. For example, a networkdevice may send its unique security key, which it may have stored alongwith its network ID after being registered on the network, to a networkdevice to which it wants to subscribe. However, subscriptions may takeon many other forms, including sending a different form ofidentification to a network device to which a network device wants tosubscribe. However, subscriptions may take on many other forms,including sending a different form of identification to a network deviceto which a network device wants to subscribe.

Upon receiving a subscription from another network device or accessdevice, the device being subscribed to may store a list of the devicesthat subscribed to it. For example, network device 1102 may store a listof network devices 1104, 1106 and 1108 and access device 108 after thosedevices subscribe to network device 1102. Then, when network device 1102undergoes a change in status, network device 1102 may send that changein status to only the devices that had previously subscribed to it butwhere the subscription had not yet expired. Furthermore, according tosome embodiments, the subscription list of a network device may beautomatically updated if that device receives notification that anotherdevice has left the range of the local area network, either from thatdevice itself or from a different device. Therefore, the various deviceswithin a given local area network, such as network 1200, each containcontinuously updated statuses of each other device on the network andobtain those statuses and updates through direct communication withoutnecessary use of the cloud.

FIG. 13 illustrates an embodiment of a process 1300 for providing aresource bundle 1332 that defines a visual interface module forcontrolling a network device. As shown, the process 1300 can involveexchanging intra-network communications 1310 and 1312 between multiplenetwork devices 102, 1102, 502, 1302 and an access device 108. Theprocess 1300 can also generate a default interface using a fundamentalmodel describing all supported interactions for a network device. Forexample, intra-network communication 1312 can be sent from networkdevice 102 to access device 108 in response to a query 1310.Communication 1312 can include information describing interactions thatnetwork device 102 supports. In some embodiments, communication 1310includes a set of attributes for each interaction type supported bynetwork device 102. When these attributes are received by access device108, access device 108 can use the attributes to form a complete andfundamental model describing all interactions for network device 102.This fundamental model can be used by access device 108 to determine abasic, default visual interface for network device 102. For instance,such a model can be used to create a default visual interface in caseswhere a resource bundle for network device 102 is otherwise unavailable(e.g., when a resource bundle 1332 has not been developed for networkdevice 102 and/or access device 108 cannot connect to cloud network 114to obtain resource bundle 1332). In an embodiment, the default visualinterface can be included in a known interface list 1320 that is locallystored in cache 1318 on access device 108.

In one embodiment, a resource bundle 1332 can be stored in a memory orcache of a network device, such as a cache accessible by interfacedevice 1301 of network device 102. For example, in environments whereconnectivity to cloud network 114 is unavailable, intermittent, orunreliable, an initial resource bundle 1332 stored in cache of networkdevice 102 can be transmitted directly from network device 102 to accessdevice 108. For example, this embodiment may include usingcommunications 1310 and 1312 to transmit an initial,manufacturer-supplied resource bundle 1332 directly from a networkdevice 102 to access device 108 in cases where access device 108 cannotconnect to cloud network 114.

As shown, the process 1300 may be performed by one or more computingdevices, such as network device 102, a server associated with cloudnetwork 114, and access device 108 described above with reference toFIGS. 1, 5, and 11-12. In some embodiments, the network device 102 isassociated with a home automation network, such as the local areanetwork 100 described above with respect to FIGS. 1, 5, and 11-12.Process 1300 is illustrated as a data flow diagram, the operation ofwhich represents operations that can be implemented in hardware,computer instructions, or a combination thereof. Gateway 110 isconnected to cloud network 114, and allows network device 102 to connectto the cloud network 114, the Internet, or other external networks viagateway 110. In some embodiments, the network device 102 may be a homeautomation network device that allows a user to access, monitor,control, and/or configure various home appliances located within theuser's home, such as a television, radio, light, microwave, iron, spaceheater, sensor, and/or the like. For example, the user can monitor andcontrol network devices by interacting with a visual interface of thegateway 110 (i.e., a web page for gateway 110) and/or a visual interfacerendered on display 1322 of the access device 108.

In an embodiment, an application may be run on the access device 108.The application may cause the access device 108 to use a resource bundleto present a display 1322 with a modular visual interface for eachnetwork device accessible on the local area network 100. When theapplication is run on the access device 108, the access device 108 canaccess a cache 1318.

The cache 1318 can be a local cache located in onboard storage of theaccess device 108. The cache 1318 can contain a known interface list1320 with records 1324, 13213 and 1328 including interface informationfor different, known types of network devices. The interfaces ininterface list 1320 can be defined by resource bundles, such as resourcebundle 1332. In additional or alternative embodiments, one or more ofthe interfaces in interface list 1320 can be a default interfacegenerated using a fundamental model describing all supportedinteractions for a network device. As shown, each of records 1324, 1326and 1328 can include a device type, a unique interface module ID, andcontrols information. The known interface list 1320 can include a recordfor each device known by the access device 108 to exist on the localarea network 100. When the application is run on the access device 108,the access device 108 can access the known interfaces 1320 in the cache1318 to present the display 1322, which lists modular interfaces foreach network device on the local area network 100. In an example, thedisplay 1322 can include a modular tile for each connected networkdevice having an interface in the known interface list 1320. Exemplarycommunications used to populate cache 1318 are described in thefollowing paragraphs.

The process 1300 can include exchanging communications 1336 and 1338between a third party interface developer 1330 and cloud network 114. Inresponse to a request 1336 sent from cloud network 114, communication1338 can be sent from a third party interface developer 1330 to cloudnetwork 114. In some embodiments, request 1336 is not used as part ofprocess 1300. For example, a third party developer 1330 of a visualinterface module for network device 102 may initiate communication 1338to cloud network 114 in order to transmit a resource bundle 1332 tocloud network 114. A resource bundle 1332 for a given network device candefine an interface module for that device.

In one example, communication 1338 can include a resource bundle 1332corresponding to network device 102. The resource bundle can be embodiedas a structured folder structure whose contents define all visual andinteractive elements in a tile for the network device. For example,resource bundle 1332 can be a zip file sent from a device manufactureror a third party developer 1330 that is submitted or uploaded to cloudnetwork 114. The resource bundle 1332 includes a unique device ID andfiles 1334 defining graphical content of a visual interface module. Thegraphical content can include definitions of interactive elements forthe interface module. The resource bundle can include templates defininginteractive control states for each of the interactive elements,language translations for tile text, any menus for the tile, andgraphical content of the menus. For example, the resource bundle 1332can define templates, text, and graphical content using a markuplanguage, such as HTML5. According to an embodiment, after an initialresource bundle 1332 is uploaded from developer 1330 to cloud network114 via communications 1338, a subsequent request 1336. In thisembodiment, request 1336 is a request from cloud network 114 for anupdated resource bundle 1332. If an updated resource bundle 1332 isavailable from developer 1330, then another communication 1338 is sentto upload the an updated resource bundle 1332 to cloud network 114.

The process 1300 can also include utilizing communications 1306 to sendrespective, unique identifiers for network devices 102, 1102, 502, and1302 to a server of cloud network 114. In the example of FIG. 13,network device 1102 is a fan, network device 502 is a switch, andnetwork device 1302 is a space heater. For simplicity, communications1306 are shown as direct communications between network devices 102,1102, 502, 1302 and cloud network 114. However, it is to be understoodthat in embodiments, communications 1306 can be sent from gateway 110 orrespective manufacturers of network devices 102, 1102, 502, 1302 tocloud network 114.

In the example of FIG. 13, communications 1306 include respectiveregistration information for network devices 102, 1102, 502, 1302. Forexample, communications 1306 can include respective, unique device IDsfor network devices 102, 1102, 502, 1302. In some embodiments, theregistration information may identify one or more sets of capabilitiesof network devices 102, 1102, 502, 1302. The registration informationfor each network device can include a unique identifier for the networkdevice, a default name of the network device, one or more capabilitiesof the network device, and one or more discovery mechanisms for thenetwork device.

Communications 1306 can be sent as part of the process 1300 in order totransmit an indication that network devices 102, 1102, 502, and 1302 areassociated with the network. For example, network device 102 maytransmit the indication to the server of the cloud network 114. In someembodiments, the transmitting may include transmitting a uniqueidentifier (ID) for the network device 102. For example, the networkdevice 102 may send a communication to the server indicating a uniquedevice ID for the network device 102. In such embodiments, the servermay then determine that a match between the unique device ID and a knownresource bundle 1332 exists. The cloud network 114 can include a datastore 1304 of known resource bundles 1332.

Access device 108 can request and download a resource bundle 1332 for agiven network device by exchanging communications 1314 and 1316 with thecloud network 114. Once the resource bundle 1332 is received at accessdevice 108 from the cloud network 114, the bundle can be used togenerate a visual interface module. The visual interface module can thenbe used to render a modular interface within display 1322. In anembodiment, the interface module can be stored in cache 1318 at accessdevice 108. For example, cache 1318 can be used to store a tile databasewhere each record in the database is uniquely identified by a tile ID.In alternative or additional embodiments, cache 1318 can be used tostore a known interface list 1320. As shown in the example of FIG. 13,known interface list 1320 can include records 1324, 1326 and 1328 withinterface information for different, known types of network devices.

Cloud network 114 can use a unique device ID to determine a resourcebundle 1332 that corresponds to one of the network devices 102, 1102,502, 1302. As shown in FIG. 13, cloud network 114 can access a datastore 1304 of resource bundles 1332. Each resource bundle 1332 candefine a respective visual interface module for one of the networkdevices 102, 302, 502, 1302. A plurality of uniquely identified resourcebundles 1332 can be stored in data store 1304. For example, eachresource bundle 1332 in data store 1304 can be associated with a uniqueinterface module ID. In an embodiment, data store 1304 is a databaseconfigured to store resource bundles 1332 defining modular tiles for aplurality of network devices, with each of the stored resource bundles1332 being identified by a unique ID. For instance, network device 102having a unique device identifier may be matched with an existingresource bundle 1332 based on comparing information received from thenetwork device 102 with information stored in data store 1304. In caseswhere an existing resource bundle 1332 for network device 102 is notfound in data store 1304, cloud network 114 can use information providedby network device 102 to generate a default interface module, where theinformation is provided via communication 1306 as part of a registrationprocess for network device 102. The generated, default interface modulecan then be assigned a unique interface module ID and stored in thecloud network 114. In some embodiments, an updated resource bundle 1332can be used to update an existing record stored in data store 1304.

After determining the resource bundle 1332 and/or the unique interfacemodule ID for network device 102, cloud network 114 sends communication1308 to network device 102 in order to provide a unique interface moduleID to the network device 102. In one embodiment, communication 1308 caninclude a unique tile ID corresponding to a modular tile for networkdevice 102 that is defined by a resource bundle 1332 stored in datastore 1304. In some embodiments, communication 1308 includes a uniquetile ID corresponding to a modular tile defined by a resource bundle1332. Upon receiving communication 1308 with the unique interface moduleID (e.g., a unique tile ID), the network device 102 can store the uniqueinterface module ID. For example, the unique interface module ID can bestored by an interface device 1301 of the network device 102 that isconfigured to provide the interface module ID to a mobile device orgateway. In an embodiment, the interface device 1301 is implemented as a‘smart module’ in hardware and firmware, such as, for example, a systemon a chip (SOC) integrated into the network device 102.

The interface device 1301 can include flash memory and dynamic randomaccess memory (DRAM). The flash memory may be used to store instructionsor code relating to an operating system, one or more applications, andany firmware. The flash memory may include nonvolatile memory so thatany firmware or other program can be can updated. In the event theinterface device 1301 loses power, information stored in the flashmemory may be retained. The DRAM of the interface device 1301 may storevarious other types of information needed to run the interface device1301, such as all runtime instructions or code. The flash memory or DRAMor a combination thereof may include all instructions necessary tocommunicate with network device 102.

The process 1300 can include sending, from the access device 108,intra-network communication 1310 including a query, to the networkdevice 102. The query can be a request for information such as a queryfor capabilities, a request for an identity of the network device 102,and/or a request for a unique interface module ID. For example,communication 1310 can be sent from access device 108 to network devices102, 1102, 502, 1302 to query network devices 102, 1102, 502, 1302 abouttheir respective identities and capabilities. In response to the querysent from access device 108, the process 1300 can include receivingintra-network communications 1312 at the access device 108 with deviceinformation for the network devices 102, 1102, 502, 1302. According toan embodiment, in response to the query, the network devices 102, 1102,502, 1302 can send communications 1312 to inform the access device 108of their identities and/or capabilities. For instance, in response toreceiving the query, the network device 102 may send communications 1312to the access device 108 with at least a unique interface module ID.

The process 1300 can include utilizing intra-network devicecommunications 1310 and 1312 as part of a discovery process for thenetwork device 102. For example, when the network device 102 isinitially connected to the network, it and access device 108 canautomatically exchange communications 1310 and 1312 to provide theaccess device 108 with information that can be used to determine abasic, default visual interface to be stored in cache 1318. In anembodiment, the process 1300 can generate a default interface using afundamental model describing all supported interactions for a givennetwork device. For example, intra-network communication 1312 can besent from network device 102 to access device 108 in response to a queryincluded in communication 1310. Communication 1312 can includeinformation describing interactions that a given network device (e.g.,network device 102) supports. For instance, communication 1312 canincludes a set of attributes for each interaction type supported bynetwork device 102. When these attributes are received by access device108, it can use the attributes to form a complete and fundamental modeldescribing all interactions supported by network device 102. Thisfundamental model can then be used by access device 108 to determine abasic, default visual interface for network device 102. For example,such a model can be used to create a default visual interface in caseswhere a resource bundle 1332 for network device 102 is otherwiseunavailable (e.g., when resource bundle 1332 has not been developed fornetwork device 102 and/or when access device 108 cannot connect to cloudnetwork 114 to obtain resource bundle 1332). In an embodiment, thedefault visual interface can be included in a known interface list 1320that is locally stored in cache 1318 on access device 108.

Within the context of a modular tile framework, embodiments candynamically render a functional user interface without having todownload the appropriate interface template from a remote server, suchas a server associated with the cloud network 114, in order to control anewly discovered network device. These embodiments can be used in caseswhere a connection to the Internet or the cloud network 114 isunavailable or unreliable, and immediate use of a newly discoverednetwork device is desired. In this case, an application on the accessdevice 108 or a stationary device such as gateway 110 could, based oncertain information received from the network device 102, dynamicallyrender a functional interface for immediate use. Such a functionalinterface may not be the ideal, visually optimized, interface that isdownloadable from the cloud network 114. However, such a functionalinterface will suffice until the application is able connect to theInternet and/or the cloud network 114 and subsequently download theappropriate and visually optimized interface module for the networkdevice 102.

In some embodiments, communication 1312 may be received when the networkdevice 102 is rebooted (e.g., powered on, reset or restored to defaultsettings, or the like). For example, when the network device 102 isrebooted, it may broadcast one or more messages on the local areanetwork 100 to discover whether there are any mobile devices in thelocal area network 100. For example, communication 1312 may be broadcastaccording to a UPnP protocol during a discovery process. The networkdevice 102 may receive communications 1310 from access device 108indicating that it is located within the local area network 100 andinterrogating network device 102 about its functionalities. That is,after receiving a broadcast message from network device 102, accessdevice 108 may then query network device 102 by sending communication1310 in order to receive the communication 1312 including informationabout the network device.

After receiving communication 1312, if the access device 108 can accessthe cloud network 114, it sends a communication 1314 to the cloudnetwork 114 as a request for a resource bundle 1332 for a given networkdevice, such as network device 102. Communication 1314 is described infurther detail below. Otherwise, if the access device 108 cannot accessthe cloud network 114, the access device 108 looks up the uniqueinterface module ID received from the network device 102 in cache 1318.As discussed above, cache 1318 can be a local cache stored on the accessdevice 108. Basic properties for known interfaces can be stored in cache1318 as a device type and controls information. These basic propertiescan include, for example, a default icon, a default name, andinteractive elements for controlling one or more primary functionalitiesof a network device. The primary functionalities can include, forexample, powering the network device on and off. The basic propertiescan also include controls information for secondary functionalities.

In some embodiments, when the access device is connected to the cloudnetwork 114, the access device 108 sends communication 1314 to query thecloud network 114 about a given network device (e.g., network device102). The communication 1314 can include at least the unique interfacemodule ID for the network device 102. The communication 1314 can includea query or request for a resource bundle 1332 for the network device102. In cases where access device 108 already has a resource bundle 1332for network device 102, communication 1314 can be request can be for anupdated resource bundle. At this point, the cloud network 114 cancompare the unique interface module ID of the network device 102 toknown interface module IDs stored in data store 1304 in order todetermine that there is a match between the unique interface module IDsent with communication 1314 and a resource bundle 1332. If the cloudnetwork 114 finds a resource bundle 1332 for network device 102 in itsdata store 1304, it transmits the resource bundle 1332 to the accessdevice 108 via communication 1316. For example, if the access device 108is currently using a default interface module for network device 102that was determined based on exchanging communications 1310 and 1312,and then subsequently is able to connect to the cloud network 114,communications 1314 and 1316 between the access device 108 and the cloudnetwork 114 can be used to obtain a resource bundle 1332 for the networkdevice 102. Once the resource bundle 1332 is downloaded to access device108 from cloud network 114 using communications 1316, the resourcebundle 1332 can be used to generate an updated interface module for thenetwork device 102.

For example, upon receiving communication 1316 from the cloud network114, the access device 108 can use a resource bundle 1332 to populate arecord in cache 1318 corresponding to the network device 102 with devicetype and controls information defined in resource bundle 1332. That is,when the access device 108 is remote from the local area network 100, itcan exchange communications 1314 and 1316 with the cloud network 114 toreceive a resource bundle 1332 for a network device. Informationreceived via communications 1316 can be used to populate records ofcache 1318. Records in cache 1318 can be updated using resource bundles1332 received via communication 1316. In additional or alternativeembodiments, new records can be created in cache 1318 when communication1316 includes a resource bundle 1332 for a newly discovered networkdevice.

Records 1324, 1326, 1328 in cache 1318 store device types, uniqueinterface module IDs, and controls information for known networkdevices. The access device 108 uses the records in cache 1318 to rendervisual interfaces in the display 1322. For example, the display 1322 caninclude a navigable list of modular tiles corresponding to networkdevices in the local area network 100.

Display 1322 can also include an indicator representing a state ofnetwork device 102. In embodiments, communications 1312 and/or 1314 caninclude a last known state of the network device 102 and/or historicaldata associated with the network device 102. In one embodiment, suchstate information can be based on information received via communication1312 from the network device 102 when the access device 108 is connectedto the local area network 100. In this way, display 1322 of the accessdevice 108 can reflect a current state and historical data for thenetwork device 102 when the mobile device is not connected to the localarea network 100. In additional or alternative embodiments, the stateinformation can be based on information received via communication 1316from the cloud network 114 when the access device 108 is connected tothe cloud network 114. Using the state information, an interface moduleor tile for the network device 102 within display 1322 can indicate an‘on’ or ‘off’ state for the network device 102 when the network deviceis powered on or off.

FIGS. 14-16 provide techniques for providing information used to definevisual interface modules for controlling network devices that existwithin a local area network. The techniques can provide resource bundlesdefining visual interface modules. The techniques allow a user of anaccess device to monitor and control network devices within the localarea network regardless of whether the access device has previously beenassociated with or connected to the network devices. For example, usingthe techniques described below, the server of the cloud network 114 maydetermine that a newly discovered network device is connected with thewireless local area network 100, and use a unique ID to retrieve apreviously registered resource bundle that defines a visual interfacemodule for the network device. Upon determining that a visual interfacemodule for the network device exists, the server may provide theresource bundle to an access device 108 associated with the wirelesslocal area network 100 in order to facilitate generating an interfacemodule for the network device.

Processes 1400, 1500, and 1600 are illustrated as a logical flowdiagrams, the operation of which represents operations that can beimplemented in hardware, computer instructions, or a combinationthereof. In the context of computer instructions, the operationsrepresent computer-executable instructions stored on one or morecomputer-readable storage media that, when executed by one or moreprocessors, perform the recited operations. Generally,computer-executable instructions include routines, programs,applications, objects, components, data structures, and the like thatperform particular functions or implement particular data types. Theorder in which the operations are described is not intended to beconstrued as a limitation, and any number of the described operationscan be combined in any order and/or in parallel to implement theprocesses.

Additionally, the processes 1400, 1500, and 1600 may be performed underthe control of one or more computer systems configured with executableinstructions and may be implemented as code (e.g., executableinstructions, one or more computer programs, or one or moreapplications) executing collectively on one or more processors, byhardware, or combinations thereof. As noted above, the code may bestored on a computer-readable storage medium, for example, in the formof a computer program comprising a plurality of instructions executableby one or more processors. The computer-readable storage medium may benon-transitory. The processes 1400, 1500, and 1600 can utilizeintra-network communications to inform an access device of the existenceof a network device and to provide a resource bundle defining aninterface module for the network device to the access device. In someaspects, the processes 1400, 1500, and 1600 may be performed by one ormore computing devices, such as the server of the cloud network 114, theaccess device 108, the gateway 110, and the network devices 102, 1102,1104, 1106, 1108, 1302 discussed above with respect to FIGS. 1 and11-13.

FIG. 14 shows a flow chart showing an exemplary set of steps for aprocess 1400 of generating interface modules for network devices. Theinterface modules are usable for controlling the network devices. At1402, the process 1400 includes receiving a communication including aunique identifier of a network device connected to a network. As shownin FIG. 14, this step can be performed as part of discovery of a networkdevice. In an embodiment, the communication received in 1402 can includea unique device ID for a network device. For instance, step 1402 can beperformed using communications 1310 and 1312 described above withreference to FIG. 13.

Next, at 1404, a determination is made as to whether the network deviceis known. As shown, 1404 can be performed by determining if the networkdevice discovered at 1402 is already known by an access device. If it isdetermined that the network device is known, control is passed to 1412where the unique identifier is used to identify a previously receivedresource bundle for the network device. As depicted in FIG. 14, 1412 canbe performed by identifying a resource bundle in a local cache or datastore. For example, 1408, the unique identifier received at 1402 can beused in step 1412 to identify a resource bundle for the network devicewithin data store 1304. The resource bundle can define an interfacemodule stored in local cache 1318 at access device 108. Otherwise, if itis determined that the network device is not known (e.g., has not beenseen by the access device before), control is passed to 1406.

Next, at 1406, a determination is made as to whether a connection to acloud network exists or can be established. If it is determined that aconnection to the cloud network exists, control is passed to 1414 wherea request is transmitted to the cloud network. The request transmittedin 1414 can be a request for a resource bundle defining an interfacemodule for the network device, where the request includes the uniqueidentifier received in 1402. Otherwise, if it is determined that noconnection to the cloud network exists, control is passed to 1408.

At 1408, the network device is interrogated or queried to request itscapabilities and the interaction models it supports. As shown, 1408 caninclude using the network device's capabilities and supportedinteraction models to form a fundamental model describing allinteractions supported by the network device. After querying the networkdevice and forming the fundamental model, control is passed to 1410where a generic interface module for the network device is generated.The generic interface module is generated based on the network device'scapabilities and supported interaction models obtained at 1408. Aftergenerating the generic interface module, control is passed back to 1406to determine if a connection to the cloud network can now beestablished.

At 1414, a request is transmitted to the cloud network. The requesttransmitted in 1414 can be a request transmitted to cloud network 114.The request can be a request for a resource bundle 1332 defining aninterface module for the network device. As shown in FIG. 14, therequest transmitted at 1414 includes the unique identifier received in1402. After transmitting the request, control is passed to 1416.

Next, at 1416, a determination is made as to whether a resource bundlefor the network device is available from the cloud network. If it isdetermined that a resource bundle is available from the cloud network,control is passed to 1418 where the resource bundle for the networkdevice is received from the cloud network. Otherwise, if it isdetermined that no resource bundle is available via the cloud network,control is passed to 1408.

At 1418, the process 1400 includes receiving a resource bundle for thenetwork device. In an embodiment, the resource bundle is received from acloud network 114 in response to the request transmitted at 1414. Forexample, steps 1414 and 1418 can be performed using communications 1314and 1316 described above with reference to FIG. 13. The resource bundlereceived at 1418 can be a new resource bundle for a newly discoverednetwork device or an updated resource bundle for the network device. Forinstance, in an embodiment, process 1400 can optionally perform steps1414-1418 even if it is determined at step 1404 that the network deviceis a known, previously discovered network device. In this way, a requestfor an updated resource bundle can be transmitted in step 1414 andfulfilled in step 1418 in cases where an access device has an interfacemodule for a known network device. This embodiment can be used to obtainupdated resource bundles subsequent to receiving an initial resourcebundle in the event that a third party developer 1330 or network devicemanufacturer uploads updated resource bundles to cloud network 114.

Next, at 1420, the process 1400 includes using a resource bundle togenerate a customized interface module for the network device, theinterface module being usable to monitor and/or control the networkdevice. For example, 1420 can be performed by using either a previouslyreceived resource bundle identified at 1412, or a resource bundlereceived at 1418.

Step 1420 can be performed by application invoking the resource bundleto generate the interface module for the network device. Depending onthe determination made in 1404, the resource bundle used at 1420 can bea resource bundle received from the cloud network or a previouslyreceived resource bundle identified at 1412. Step 1420 can be performedby an application executing on an access device or a gateway. When theinterface module is subsequently executed at the access device orgateway, it displays interface elements usable to monitor and/or controlthe network device.

FIG. 15 illustrates an embodiment of a process 1500 for providing aresource bundle for a network device. The resource bundle defines aninterface module usable to monitor and control the network device. Insome aspects, the process 1500 may be performed by computing devices,such as access device 108, the gateway 110, and the server of the cloudnetwork 114 discussed above with respect to FIGS. 1 and 11-13.

The process 1500 utilizes intra-network and inter-network communicationsto inform a cloud network server of the existence of a resource bundlefor a network device and to provide the resource bundle defining aninterface module for the network device to an access device and/or agateway. At 1502, the process 1500 includes receiving a communicationindicating that a resource bundle is available. The resource bundle canbe for a network device that has been detected in a network. As shown inFIG. 15, 1502 can include receiving a communication including a resourcebundle, registration information for the resource bundle, and a uniqueID for a network device interface. In an embodiment, 1502 can includereceiving the communication with a resource bundle 1332 from a thirdparty developer 1330 or a network device manufacturer. For example, 1502can be performed using communications 1336 and 1338 described above withreference to FIG. 13.

At 1504, the resource bundle received at 1502 is registered and saved.As shown, this can include registering the resource bundle in acloud-based network device interface registry and saving the resourcebundle in a data store. In one embodiment, the resource bundle is savedin data store 1304 in cloud network 114. After the resource bundle isregistered and saved, control is passed to 1506.

At 1506, a communication is received, the communication including arequest for a resource bundle for a network device. As shown, therequest can include a unique ID for the network device. In anembodiment, 1506 can be performed using communication 1314 describedabove with reference to FIG. 13. After the communication is received,control is passed to 1506 where the unique ID is used to determine aresource bundle for the network device. In the example of FIG. 15, 1506can include identifying the latest, most-recent resource bundle for thenetwork device.

Next, at 1510, the resource bundle identified at 1508 is transmitted. Inan embodiment, step 1510 can include transmitting a resource bundledefining an interface module for a network device to an access device108 and/or a gateway 110 associated with a given network.

The process 1500 can further includes displaying, within an applicationexecuting on the access device and/or gateway, a tile defined by thetransmitted resource bundle. This can be accomplished by rendering thedisplay 1322 described above with reference to FIG. 13. For example, thetile can be configured to accept input corresponding to a selection of asetting for a primary function for the network device. In an embodiment,this can include receiving a touch input at an interactive element of atile for the network device. Then, the process 1500 can includedetermining whether a secondary function is related to the primaryfunction. If it is determined that there is a related secondaryfunction, a submenu, such as, for example, a peekaboo drawer, can bedisplayed with options for the secondary function. In an embodiment,process 1500 can include automatically displaying the submenu andindicating a default selection for the secondary function. For example,if the primary function selected turned on a space heater network device1302, process 1500 can include displaying low, eco, medium, and highsecondary operational modes for the space heater and indicating that adefault setting of eco mode has been selected.

FIG. 16 is a flowchart illustrating a process for using resource bundlesto generate interface modules for network devices. FIG. 16 illustratesan embodiment of a process 1600 for providing a resource bundle for anetwork device. The resource bundle defines an interface module usableto monitor and control the network device. In some aspects, the process1600 may be performed by computing devices, such as access device 108,the gateway 110, and the server of the cloud network 114 discussed abovewith respect to FIGS. 1 and 11-13.

The process 1600 utilizes intra-network communications to inform a cloudnetwork server of the existence of network devices and to provideresource bundles defining corresponding interface modules for each ofthe network devices to an access device and/or a gateway. At 1602, theprocess 1600 includes receiving communications indicating that networkdevices have been detected in a network. As shown in FIG. 16, 1602 canbe performed as part of device discovery.

At 1604, it is determined that respective resource bundles areassociated with respective ones of the plurality of network devices. Therespective resource bundles can define visual interface modules for therespective ones of the discovered network devices. This can beaccomplished by looking up unique interface IDs in a data store 1304 ofresource bundles. Next, at 1606, the resource bundles corresponding tothe network devices are transmitted. In an embodiment, step 1606 caninclude transmitting resource bundles to an access device 108 and/or agateway 110 associated with a given network.

At 1608, the process 1600 includes using the resource bundles togenerate interface modules for the network devices. As shown, 1608 caninclude generating and/or updating interface modules by using respectiveresource bundles corresponding to respective ones of the plurality ofnetwork devices. In an embodiment, the resource bundles can be invokedwithin an application executing on an access device and/or gateway inorder to generate tiles corresponding to the network devices. The tilescan be rendered by the application in the display 1322 described abovewith reference to FIG. 13.

Once any of the above techniques are used to provide a resource bundledefining an interface module corresponding to a network device, theinterface module can be executed by an application executing on a mobiledevice in order to allow the mobile device to monitor and control thenetwork device. The server may store unique identifier (e.g., networkdevice MAC address) of a first network device 102 and a correspondinginterface module ID in a first logical record of a data store 1304 ofinterfaces at a first memory space. The server may also store the uniqueidentifier of a second network device 104 and its correspondinginterface module ID in a second record of data store 1304 at a secondmemory space. In some embodiments, the data store 1304 of interfacesincludes pointers. For example, a pointer stored with each record maypoint a program to a memory location where code of a resource bundledefining an interface module is stored. In an embodiment, the code canbe in a markup language, such as, for example, HTML5. In certainembodiments, the code defines a modular tile having an icon, a name, andone or more interactive elements useable to control a network device. Bystoring pointers to resource bundles, all of the information needed toobtain resource bundle is accessible from the data store. For example,using a pointer, code in a resource bundle defining an interface usableto control network devices can be obtained and provided to an accessdevice.

Another technique for providing an interface for controlling a networkdevice may include using an existing, default interface in cases where aresource bundle is unavailable or is not readily obtainable. Forexample, this technique may include determining that a resource bundlefor a network device does not exist, creating a new, default interfacebased on information about the network device, and designating the newinterface as the interface to be used for controlling the networkdevice. A new record may then be created including a unique interfaceidentifier for the newly created interface. The access device may storethe new interface in cache 1318 of interfaces.

Yet another technique for providing an interface for controlling anetwork device may include storing a resource bundle in a memory orcache of a network device. For example, in environments where cloudconnectivity is unavailable or is not reliable, an initial resourcebundle stored on a network device can be transmitted directly from thenetwork device to an access device. For example, this technique mayinclude using communications 1310 and 1312 described above withreference to FIG. 13 to transmit a manufacturer-supplied resource bundle1332 directly from a network device 102 to an access device 108.

Once any of the above techniques are used to identify a resource bundledefining an interface module, the server may maintain a cloud-basedregistry of stored resource bundles in order to enable access devicessuch as mobile devices and gateways to control network devices by usinginterfaces defined by the resource bundles.

FIGS. 17-19 illustrate graphical interfaces, according to embodiments ofthe present disclosure. In particular, FIGS. 17-19 illustrate exemplaryembodiments of user interface (UI) displays for an application on anaccess device. In some embodiments, the access device can be a mobiledevice, such as, for example, a smart phone or a tablet device. Thegraphical interfaces depicted in FIGS. 17-19 are described withreference to the embodiments of FIGS. 1 and 11-16. For example, portionsof the visual interfaces illustrated in FIGS. 17-19 corresponding tospecific network devices can be defined by resource bundles for thenetwork devices. Also, for example, other portions of the interfaces ofFIGS. 17-19 corresponding to other network devices can be defaultinterfaces generated based on respective fundamental models describingsupported interactions for those other network devices. However, thegraphical interfaces of FIGS. 17-19 are not limited to the exampleembodiments described above with reference to FIGS. 1 and 11-16.

In an embodiment of the invention, the visual interfaces illustrated inFIGS. 17-19 are displayed on a mobile computing device, which can have atouch sensitive (i.e., touch screen) display device. For ease ofexplanation, the monitoring and control operations discussed below withreference to FIGS. 17-19 are described in the context of an applicationexecuting on an access device 108 with a touch-screen display device.However, the operations are not intended to be limited to the exemplarydevice shown in FIGS. 17-19. It is to be understood that the userinterfaces illustrated in the exemplary embodiments of FIGS. 17-19 canbe readily adapted to be rendered on displays of a variety of computingdevice platforms running a variety of operating systems.

Throughout FIGS. 17-19, displays are shown with various tiles,interactive elements, icons, links, command regions, windows, toolbars,submenus, drawers, and buttons that are used to initiate action, invokeroutines, monitor network devices, configure network devices, controlnetwork devices, or invoke other functionality. The initiated actionsinclude, but are not limited to, displaying a current state of a networkdevice, displaying historical data associated with a network device,displaying a last known status of a network device, selecting one ormore primary settings for a network device, selecting one or moresecondary or tertiary settings of a network device, and other inputs andgestures. For brevity, only the differences occurring within thefigures, as compared to previous or subsequent ones of the figures, aredescribed below.

FIG. 17 shows an exemplary display 1700 of an application executing on amobile device. The display 1700 includes modular tiles 1702A, 1702B, and1702C for interacting with network devices in a network. Theinteractions can include monitoring and controlling the network devices.The tiles 1702A, 1702B, and 1702C each include respective icons 1704A,1704B, and 1704C, and respective names 1706A, 1706B, and 1706Cidentifying different network devices. For example, tile 1702Acorresponds to a switch and shows an interactive element depicted as apower button 1708A for turning the switch on and off. Similarly, tile1702C shows a power button 1708C for turning a lamp on and off. In someembodiments, one or more of tiles 1702A, 1702B, and 1702C in display1700 can be defined by respective resource modules corresponding to thenetwork devices the tiles are associated with. In additional oralternative embodiments, one or more of tiles 1702A, 1702B, and 1702Ccan be determined using a fundamental model describing all interactionsfor the network devices the respective tiles are associated with. Forexample, if a resource bundle is unavailable for the lamp associatedwith tile 1702C, a fundamental model for the lamp can be used todetermine default tile 1702C with name 1706C (e.g., ‘lamp’), lamp icon1704C, power button 1708C, and brightness control 1710C.

Display 1700 is a visual interface usable to monitor and control one ormore network devices. Display 1700 includes modular tiles 1702 fordiscovered network devices, where the tiles 1702 each include an icon1704 representing a network device and a name 1706 for the networkdevice. In the example of FIG. 17, display 1700 includes tiles 1702A,1702B, and 1702C corresponding to three different network devices. Inparticular, tile 1702A is configured to control a switch network device.Tile 1702A includes an icon 1704A representing the switch and a name1706A for the switch. In embodiments, tiles 1702 are customizable bydevelopers and/or manufacturers of the network devices. For, example,name 1706A is customized to indicate the name of a specific type ofswitch (e.g., ‘Hall switch’). Tile 1702A also includes an interactiveelement 1708A that can be selected to control the switch. For example,interactive element 1708A can be selected with a click or press to turnthe switch on and off. Tile 1702B includes an icon 1704B for a sensornetwork device. Tile 1702B includes an icon 1704B representing thesensor and a name 1706B for the sensor (e.g., ‘My Motion Sensor’). Inembodiments, tiles 1702 are customizable by developers and/ormanufacturers of the network devices to display a current state orstatus of network devices. For, example, tile 1702B is customized toindicate a sensor-specific status (e.g., ‘Motion sensed at 11:05 AM’).In additional or alternative embodiments, tiles 1702 can convey statusinformation about a network device, including, but not limited to, afirmware version, a last known firmware update status, connectivity tocloud status, registration status (i.e., an indication that the networkdevice has a key or does not), a primary mode of the network device(i.e., on, off), a secondary mode of the device (i.e., standby, high,low, eco), a schedule, and settings for the device (see, e.g., schedule1934 in FIG. 19).

Depending on the type and capabilities of a network device, its tile1702 can display a status or state 1710 of the device. For example, FIG.17 shows how status 1710B indicates when a motion sensor last detectedmotion. In an embodiment, the display 1700 is flexible and candynamically display the most appropriate tile 1702 for whateverenvironment the application is run in. For instance, the display 1700enables a mechanism wherein a user can pre-configure a tile 1702B todisplay the most appropriate message for a connected sensor. Forexample, a user can connect any one of a plurality of types of sensors(e.g., water, humidity, temperature/heat, air pressure, light, sound,smoke, carbon monoxide, etc.) to a network. As shown in FIG. 17, insteadof presenting a default interface stating a generic message like ‘thesensor has detected something at a certain time’ the tile 1702B can beconfigured it to provide a sensor-specific status (e.g., ‘Motion sensedin the living room 11:05 AM’ or ‘someone is in the living room.’).Similarly, tiles 1702 for other types of sensor devices can beconfigured to present sensor-specific states 1710 or status messagessuch as, for example, ‘the leak detector at a certain location hasdetected elevated levels of moisture’ or ‘the smoke detector hasdetected smoke in the kitchen.’

As shown in FIG. 17, the status 1710C in the tile 1702C for the lampdevice can indicate a brightness level for the lamp. In the exemplarydisplay 1700, status 1710C indicates that the lamp is at 0%, whichcorresponds to the lamp being turned off. In an embodiment, this statusis also reflected by the power button 1708C for the lamp not being litup or shaded/bolded. When a network device has extended capabilities,such as secondary or tertiary functionalities, an interactive element1712 can be selected to expand and contract a menu includingcontrollable settings for the capabilities. The menu can be, forexample, a full drop down menu or drawer with interactive elements forsetting the extended capabilities of the network device. Examples ofsuch full menus and drawers are provided in FIG. 19, which is discussedbelow.

FIG. 17 also shows how display 1700 can include selectable icons andlinks 1714, 1716, 1718, 1720, 1722, and 1724 outside of the tile displayarea. In the example of FIG. 17, refresh icon 1714 can be selected torefresh information presented in display 1700, such as, for example,status and state information displayed in tiles 1702A, 1702B, and 1702C.For instance, the status 1710B in tile 1702B for the motion sensor canbe refreshed on an automatic, periodic basis, in addition to beingmanually updated when refresh icon 1714 is selected. Similarly, thebrightness status 1710C in tile 1702C for the lamp can be updated whenrefresh icon 1714 is selected. The edit link 1716 can be selected toedit the list of tiles 1702A, 1702B, and 1702C. For example, edit link1716 can be selected to sort or re-order the sequence of tiles 1702A,1702B, and 1702C displayed in display 1700. Edit link 1716 can also beselected to delete one of the tiles 1702A, 1702B, and 1702C in caseswhere a user no longer wants to view a given tile. Devices icon 1718 canbe selected to list discovered network devices in a network.

Rules icon 1720 can be selected to display rules pertaining to networkdevices. For example, rules icon 1720 can be selected to display a rulethat turns on the lamp of tile 1702C for a specified duration when themotion sensor of tile 1702B detects motion. That is, by selecting rulesicon 1720, a user can create or edit a rule that turns on a porch lightfor a certain number of minutes when a motion sensor detects motion nearthe porch. In this way, rules can relate functionalities of multiplenetwork devices to each other. News icon 1722 can be selected to reviewnews items, such as news associated with network devices and/or theapplication. For instance, news icon 1722 can be selected to viewannouncements and news items relevant to network devices controlled viatiles 1702A, 1702B, and 1702C and/or information relevant to theapplication, such as notifications of available tile updates. The moreicon 1724 can be selected to access additional features of theapplication.

FIG. 18 shows another exemplary display 1800 of an application executingon a mobile device. Display 1800 shows how the status of a space heatercan be conveyed within tile 1802A. For example, tile 1802A can show thatthe heater is turned by coloring, animating, or shading a portion ofpower button 1808A. In one embodiment, a portion of power button 1808Acan be shaded green or bolded to indicate that the space heater has beenturned on. FIG. 18 also shows how a submenu 1814 can be exposed toselect a secondary setting of the heater once power button 1808A isselected. The submenu 1814 can include settings related to the primaryfunction of turning the network device on. In the example of FIG. 18,submenu 1814 is a drawer that shows secondary settings corresponding tohigh, low, and eco operational modes for the space heater when the powerbutton 1808A is selected to turn the space heater on. Because submenu1814 is typically not displayed and may only be momentarily displayedwhen the power button 1808A is selected, the submenu 1814 can beconceptualized as a ‘peekaboo drawer.’ For example, submenu 1814 may betemporarily displayed within the context of the heater being turned on.The submenu 1814 may include a default choice for a secondary functionspace heater (i.e., eco mode), and can also be used to select othersecondary functions (e.g., high or low). In this way, with a singleinput of selecting or pressing power button 1802A, both primary andsecondary functionalities can be set via a single input. Additionalfunctionalities and capabilities of the space heater can be displayed byselecting interactive element 1812 to expand a full menu. One example ofthe full menu is shown in FIG. 19, which is described below.

FIG. 19 shows another exemplary display 1900 of an application executingon an access device. In the example of FIG. 19, the status indicator1810 shows that the space heater is in standby mode. As shown in FIG.19, this standby status can be visually reinforced by the power button1808A being lightly shaded. In some embodiments, the status indicator1810 and power button 1808A can transition when the status of the spaceheater is changed. For example, when the power button 1808A is selectedto turn on the space heater, and secondary mode of ‘low’ is selected insubmenu 1814, the power button 1808A can become more fully illuminatedas compared to FIG. 19. Additionally, status indicator 1810 cantransition from ‘Eco’ as shown in FIG. 18 to ‘High’, ‘Low’ or ‘Standby’(e.g., as shown in FIG. 19) to reflect selections in submenu 1814 and/ora full menu 1926. An exemplary full menu 1926 for a space heater networkdevice is described in the following paragraph.

FIG. 19 also shows how detailed status and extended functionalities ofthe space heater network device can be conveyed within a full menu 1926accessed via tile 1802A. In particular, full menu 1926 can be displayedas drawer when interactive element 1812 is selected. In the example ofFIG. 19, the full menu 1926 can include interactive elements 1930, 1932,1936 for selecting settings or actions corresponding to tertiary orextended functions of the space heater. In particular, the exemplaryfull menu 1926 is shown as a drawer displaying room and externaltemperature readings 1928, thermostat setting 1930, an auto off setting1932, schedule information 1934 for the space heater, and a frostprotection setting 1936. In the example of FIG. 19, the auto off setting1932 is a selectable duration (i.e., a number of hours). In embodiments,the auto off setting 1932 can be changed via a scroll wheel interfaceelement and/or directly entered as text. Full menu 1926 can beimplemented as a drawer showing historical data associated with anetwork device, such as, for example, past temperature readings 1928 andpatterns of temperature readings 1928. The historical data may be usedto determine tertiary settings or functionalities, such as, for example,the frost protection setting 1936. As shown, the frost protectionsetting 1936 can be toggled on an off within full menu 1926. In theexemplary display 1900, the space heater will automatically turn on ifthe room temperature falls below 40 degrees Fahrenheit. Certain extendedcapabilities can be tied to rules. For example, the rule for frostprotection can be changed by selecting rules icon 1720 in order to editthe logic for the rule.

As shown in the embodiments of FIGS. 17-19, the status or state of anetwork device can be indicated within the tile with text and/orgraphically. For instance, coloring or shading of an interactiveelement, such as power button 1808A, can indicate whether a networkdevice is connected to the network, powered off, on, restarting, or in astandby, sleep or hibernation mode. In accordance with these examples,part of the power button 1808A in the tile 1802A can be shaded greenwhen the space heater is powered on, red or un-shaded when the spaceheater is off, and yellow or lightly shaded when the space heater is instandby mode. In additional or alternative embodiments, a power buttoncan pulsate or slowly blink when the device is in a transitional state.Examples transitional states include booting up, restarting, connectingto a network, or re-connecting to the network.

In some embodiments, the displays of FIGS. 17-19 can be rendered byperforming a computer-implemented method. For example, the method caninclude receiving, at a computing device, one or more interface modules,wherein each interface module defines a tile associated with acorresponding network device on a network, and then presenting, on adisplay of the computing device, a graphical interface including the oneor more tiles, wherein each tile includes information identifying thecorresponding network device and one or more interactive elements usableto control the network device. For example, the computing device can beaccess device 108, the display can be display 622, and the tiles caninclude tiles 1702A, 1702B, 1702C and 1802A. The one or more interactiveelements can correspond to a primary setting or function of the networkdevices, such as, for example, power buttons 1708A, 1708C and 1808A. Inembodiments, the information identifying the network devices can includean icon, name, and state for each of the network devices. For example,the information identifying the network devices can include icons 1704A,1704B, 1704C and 1804A, and corresponding device names 1706A, 1706B,1706C, and 1806A. The information identifying a state of a networkdevice can be status indicators 1710B, 1710C, and 1810. In someembodiments, the method can also include receiving, at the computingdevice, input corresponding to a selection of a primary setting for thenetwork device, and the tile can include a contextual menu or drawer,such as submenu 1814, corresponding to a secondary setting orfunctionality for the network device, wherein the secondary setting orfunctionality is associated with the primary setting or functionality.According to certain embodiments, a tile 1802A associated with a networkdevice having extended or tertiary functionalities includes a full menuor drawer 1926, wherein the full menu 1926 is displayed in response toreceiving input corresponding to a selection of interactive element1812. For example, the full menu 1926 can include interactive elements1930, 1932, 1936 for selecting settings or actions corresponding totertiary or extended functions of the network device. The full menu 1926can be embodied as a drawer including a status indicator 1710B, 1710C,1810 for the network device, schedule data 1934, and historical dataassociated with the network device, such as, for example, temperaturereadings 1928.

In embodiments, the display device used to display the user interfacesshown in FIGS. 17-19 may be the display output device 1614 describedbelow with reference to FIG. 16. According to embodiments, user inputscan be received at the touch screen displays shown in FIGS. 17-19 viathe input device 1616 described below with reference to FIG. 16.However, alternative and additional input devices can be used, such as afinger, a stylus, a mouse, a keyboard, a keypad, a joy stick, a voiceactivated control system, or other input devices used to provideinteraction between a user and an application. As described below withreference to FIGS. 17-19, such interaction can be used to indicate anetwork device to be monitored and/or controlled, to navigate throughcontextual menus associated with a network device, and make selectionsin the menus.

FIG. 20 shows that the interface 2005 may include interactive elements(e.g., selectable icons and links) 2014, 2016, 2018, 2020, 2022, and/or2024 outside of the modular tiles 2002, 2004, 2006. For example, refreshicon 2014 may be selected to refresh information presented in interface2005, such as, for example, status and state information displayed inthe modular tiles 2002, 2004, 2006. For instance, one or more of thestatuses for each of the modular tiles 2002, 2004, 2006, respectively,may be refreshed on an automatic, periodic basis, in addition to beingmanually updated when refresh icon 2014 is interacted with (e.g.,selected). In some embodiments, interaction with refresh icon 2014 maycause interface 2005 to be updated with modular tiles corresponding tonetwork devices identified on a network (e.g., the network 100). Thenetwork devices identified on a network may be determined by an accessdevice that displays the interface 2005. The access device may receiveinformation from another network device, e.g., the gateway 104. Suchinformation may include device information about network devicesdiscovered on a network. Described below with reference to FIGS. 22-25are techniques for caching information (e.g., device information) toimprove responsiveness of an access device to present information aboutnetwork devices discovered on a network.

In some embodiments, the interface 2005 may include an edit link 2016.Interaction with the edit link 2016 may enable one or more of themodular tiles 2002, 2004, 2006 to be editable. For example, edit link2016 may be selected to sort or re-order the sequence of the modulartiles 2002, 2004, 2006 displayed in the interface 2005. Upon interactionwith the edit link 2016, one or more of the modular tiles 2002, 2004,2006 may be interacted with to be disabled or removed. Devices icon 2018may be selected to cause the interface 2005 to be displayed with one ormore network devices that are discovered in a network.

In some embodiments, the interface 2005 may include a rules icon 2020.Interaction with a rules icon 2020 may enable a graphical interface thatenables customization of display rules pertaining to the networkdevices. The graphical interface may provide one or more interactiveelements to control one or more rules related to control of the networkdevices (e.g., the network devices 302, 304, 306). In some embodiments,interaction with the rules icon 2020 may enable a graphical interface toenable a user to configure rules, modes, notifications, additionalactions, or combinations thereof. The graphical interface may bepresented in the interface 2005 or may be shown separately from theinterface 2005. An embodiment of such a graphical interface presentedupon interaction with the rules icon 2020 is described with reference toFIG. 21.

In some embodiments, the interface 2005 may include a news icon 2022.Interaction with the news icon 2022 may enable a graphical interface tobe display which presents news including information associated with thenetwork devices and/or the applications. For instance, interaction withthe news icon 2022 may cause a graphical interface to be displayed inthe access device to present information related to the network devicescontrolled via the modular tiles 2002, 2004, 2006 and/or informationrelevant to the application, such as notifications of available tileupdates.

In some embodiments, the interface 2005 may include a more icon 2024.Interaction with the more icon 2024 may provide one or more additionalgraphical interfaces to enable access to additional features forcontrolling and/or monitoring the network devices in a network. Forexample, an additional graphical interface may be displayed to controlone or more settings and/or attributes of any network devices connectedto a network. In another example, an additional graphical interface maybe displayed to configure secondary options to be initiated by aninterface used to control a network device.

FIG. 21 is an illustration of an interface 2105 for displaying rules forcontrolling operation of network devices in accordance with someembodiments. The interface 2105 may be displayed by a device, e.g., theaccess device 108. The interface 2105 may include or display a graphicalinterface. The interface 2105 may include references to elementsdescribed with reference to FIG. 20. In the example shown in FIG. 21,the interface 2105 is presented in response to interaction with theinteractive elements 2020. The interface 2105 may present rulesinformation associated with one or more network devices, e.g., any ofthe network devices 302, 304, 306, 308 or any of the network device 502,504. The interface 2105 may be presented as part of an application. Theinterface 2105 may be presented when an application is started. In someembodiments, the interface 2105 may present information associated witha network device when the network device is discovered on a network(e.g., the local area network 300). In some embodiments, the informationmay be presented in the interface 2105 upon receiving input indicatinginteraction with the interactive element 2020 (“Rules”). As will bedescribed below, the interface 2105 may be presented based oninformation received from a computing device, such as a computing devicedescribed with reference to FIG. 22. The information presented in theinterface 2105 may be received from the computing device rather thanother network devices connected to the network. The computing device mayprovide the information in response to a request by the access device ormay provide the information without a request from the access device. Indoing so, the computing device may reduce communication time for theaccess device to receive current information, thereby enabling theaccess device to improve responsiveness for presenting the currentinformation associated with network devices in the interface 2105.

In certain embodiments, the interface 2105 may present one or moregraphical interfaces with information indicating one or more criterion(e.g., a rule) related to operation of one or more devices. For example,the interface 2105 may display modular tiles 2102, 2104, 2106, eachcorresponding to a one or more criterion related to operation of one ormore network devices. The modular tile 2102, modular tile 2104, and themodular tile 2106 may correspond to a rule for operation the networkdevice 302, the network device 304, and the network device 306,respectively. For example, the modular tile 2102 may display criteriaindicating that the network device 302 is to be turned on between 8 amand 8 pm daily. In another example, the modular tile 2104 may displaycriteria indicating that the network device 304 is to be turned off on aparticular date (e.g., July 26^(th)). In another example, the modulartile 2106 may display criteria indicating that the network device 306 isto be turned on at sunset.

The information presented in the modular tiles 2102-2106 may bedetermined based on information stored by the access device. Updates tosuch information may be received from a computing device, e.g., acomputing device described with reference to FIG. 22. The modular tiles2102-2106 may be updated or deleted and new modular tiles may be addedbased on updates to rules stored by the access device. In someembodiments, the interface 2105 may include interactive elements toconfigure the rule corresponding to each of the modular tiles 2102-2106.The interface 2105 may include interactive elements to create new rules.

Now turning to FIG. 22, a block diagram of a computing device 2200 forcaching information associated with network devices is shown inaccordance with some embodiments. In particular, the computing device2200 may store information associated with network devices (e.g., thenetwork device 302, the network device 304, the network device 306,and/or the network device 308) in cache 2230.

The computing device 2200 may store information 2232 in the cache 2230.The information 2232 may be associated with one or more of the networkdevices. The information 2232 may include one or more criterion 2240related to operation of one or more network devices (e.g., a networkdevice). A criterion 2240 may include a criterion for operating one ormore devices (e.g., an “operation criterion”). For example, an operationcriterion may indicate a value for an attribute, a setting, a mode, orthe like related to operation of a network device. The criterion 2240may include a criterion indicating an instruction for controllingoperation of one or more network devices (e.g., an “instructioncriterion”). For example, an instruction criterion may indicateconditions for applying an operation criteria. A rule may be based on aninstruction criterion and an operation criterion. The conditions may bebased on time, date, weather, geographical location, an event, orcombinations thereof. The information 2232 may include deviceinformation 2250 (e.g., “Device List”). The device information 2250 mayinclude information corresponding to each network device connected to anetwork. For example, the device information 2250 may include a devicelist. The network may be a network to which the computing device 2200 isconnected. In some embodiments, the device information 2250 may includean identifier (e.g., a device identifier) corresponding to each networkdevice on the network, an device address (e.g., a device IP address),other information about the network device, or combinations thereof. Thedevice information 2250 may include status information corresponding toeach network device. The information 2232 may include status information2260 related to a status of one or more network devices connected to anetwork. The status information 2260 may be based on status datareceived from one or more network devices. The status information 2260may include other status information about network devices, e.g.,current information related to operation of each network device. Forexample, the status information may include a current value of a settingof a network device, a current value of an attribute of a networkdevice, a current state of a network device, a current mode of a networkdevice, a current value of a configuration of a network device, orcombinations thereof. The information 2232 may include other information2270 related to one or more network devices connected to a network. Suchinformation 2270 may include registration information for a networkdevice connected to a network. The other information 2270 may includeinformation communicated between network devices.

In certain embodiments, the computing device 2200 may include or beimplemented as a network device (e.g., the network device 102), arouter, a gateway (e.g., the gateway 110), a hub device, a repeaterdevice, a range extender, an access point, a bridge, a base station, aNode B device, eNode B device, an access point base station, a Femtocelldevice, a modem, any other device that provides network access among oneor more computing devices and/or external networks, the like, or acombination thereof. The computing device 2200 may include a homeautomation device such as one that may be included in the network device102 or the network device 302. The computing device 2200 may include anIoT device. The computing device 2200 may include any human-to-machineinterface with network connection capability that allows access to anetwork. For example, the computing device 2200 may include astand-alone interface (e.g., a cellular telephone, a smartphone, a homecomputer, a laptop computer, a tablet, a PDA, a wearable device such asa smart watch, a wall panel, a keypad, or the like), an interface thatis built into an appliance or other device e.g., a television, arefrigerator, a security system, a game console, a browser, or thelike), a speech or gesture interface (e.g., a Kinect™ sensor, aWiimote™, or the like), an internet of things (IoT) device interface(e.g., an Internet enabled appliance such as a wall switch, a controlinterface, or the like). The computing device 2200 may be configured toexecute an application, a web browser, a proprietary program, or anyother program executed and operated by a computing device.

In certain embodiments, the computing device 2200 may be implemented asone or more functional blocks or modules configured to perform variousoperations for caching information associated with network devices(e.g., the network device 302, the network device 304, the networkdevice 306, and/or the network device 308). The functional blocks andmodules may be implemented in hardware, firmware, software, orcombinations thereof. The computing device 2200 may include a versionmanager 2212, an information manager 2214, a communication module 2216,a rules manager 2218, a request handler 2220, and a resource determiner2220.

The communication module 2216 may support communication with otherdevices, such as a network device (e.g., one or more of the networkdevices 302-308). The communication module 2216 may include one or morewireless transceivers operable to receive and transmit wireless signals(e.g., signal 2210). The wireless signals may be received andtransmitted via an antenna of the computing device 2200. Thecommunication module 2216 may communicate with other devices via a localarea network. The local area network may include a wireless network, awired network, or a combination of a wired and wireless network. Thecommunication module 2216 may provide network access to other networkdevices. The network access may be of any type of network familiar tothose skilled in the art that can support data communications using anyof a variety of commercially-available protocols. In some embodiments,the wireless network may be any wireless network such as a wirelesslocal area network (e.g., local area network 100), such as WiFi, a PAN,such as Bluetooth® or Zigbee®, or a cellular network (e.g. a GSM, WCDMA,LTE, CDMA2000 network). For example, the communication module 2216 mayprovide wireless communication capabilities for a local area networkusing a communications protocol, such as WiFi™ (e.g., IEEE 802.11 familystandards, or other wireless communication technologies, or anycombination thereof). Using a communications protocol(s), thecommunication module 2216 may support radio frequencies on whichwireless enabled devices in the local area network can communicate.

The communication module 2216 may be configured to enable devices (e.g.,the network device 302, the network device 304, the network device 306,and/or the network device 308) to communicate with each other in a localarea network (e.g., the network 300). In some embodiments, thecommunication module 2216 may provide devices with access to one or moreexternal networks, such as a cloud network (e.g., the cloud network114), the Internet, and/or other wide area networks.

In some embodiments, the communication module 2216 may communicatedirectly with a network device (e.g., any of the network devices 102-106or any of the network device 302-208). For example, the communicationmodule 2216 may enable the computing device 2200 to communicate directlywith a network device using Zigbee™ signals, Bluetooth™ signals, WiFi™signals, infrared (IR) signals, or the like. In some embodiments, thecommunication module 2216 may communicate with a network device and/or acloud network.

In some embodiments, the communication module 2216 may send and receiveunicast transmissions. For example, the communication module 2216 maysend a unicast transmission directly to another device. The unicasttransmission may include a request for information, such as theinformation 2232, associated with a device. The device may be identifiedusing the device information 2250 corresponding to the device to whichthe communication is directed. In another example, the communicationmodule 2216 may receive a unicast transmission from another device. Aunicast transmission may be routed through another network device, suchas the gateway 110. The communication module 2216 may support unicasttransmission using UDP, TCP, HTTP, HTTPS, or other suitable protocol.

In some embodiments, the communication module 2216 may send and receivetransmissions broadcasted via a network to which the computing device2200 is connected. The transmissions may include multicast transmissionsand/or broadcast transmissions. The communication module 2216 maycommunicate using UDP, UPnP, or other suitable protocol.

The request handler 2220 may classify communication with other devices,e.g., transmissions that are received from other devices. A transmissionmay be classified based on a type of communication protocol used for thetransmission. For example, the request handler 2220 can classify atransmission as a broadcast request or multicast request based ondetermining that the transmission was sent using a protocol supportingmulticast transmission. In some embodiments, the request handler 2220may process data received via a transmission to determine whether thedata includes a request and if so, classifies the request. Data receivedin a transmission may include information indicating a request, in whichcase the request handler 2220 can determine that a request is beingmade. In some embodiments, the request handler 2220 can determine a typeof request based on a communication protocol by which the request wasreceived. For example, a request may be identified as a unicast requestby being received via a unicast transmission.

The request handler 2220 may process a data received via a transmissionto further determine details about a request included in the data. Forexample, the request handler 2220 can determine whether a request iscommunicated to obtain information (e.g., a discovery request) orwhether the request communicated to disseminate information, such as acriterion related to operation of a device, device information, statusinformation, other information, or combinations thereof. The requesthandler 2230 may inspect a received request to determine details basedon the type of request. The request handler 2220 may process a requestthat is sent via unicast transmission to determine whether the unicastrequest includes information intended for a recipient of the unicastrequest. In some embodiments, a request may include type informationindicating a type of information requested, such as a criterion relatedto operation of a device or device information. A request may includeversion information (e.g., a version identifier) that indicates aversion of the information currently stored by the device that sent therequest. In some embodiments, a request from a device may includeinformation currently stored on a device that sent the request. Suchinformation may be used to determine differences from the information2232 stored locally on the cache 2230. The type of information or theversion information may be used by the request handler 2220 to determinewhether the request is to obtain additional information, such as updatesto current information, if any.

The request handler 2220 may determine whether data received by thecomputing device 2200 should be communicated to other devices on anetwork to which the computing device is connected. In some embodiments,the request handler 2220 may determine whether data received via thecommunication module 2216 includes a request for information associatedwith a device. The request handler 2220 may use the classification of atransmission to detect a request for information. Data may indicate atype of a request and/or may include information indicating informationsought, e.g., criteria related to operation of devices, deviceinformation, or status information. The data may indicates a version orother identifier of information sought by the request. In someembodiments, based on the type of communication, the request handler2220 can determine whether received data includes information (e.g., acriterion related to operation of a device or device information) to bedistributed to the computing device or other devices. For example, basedon determining that data received in a unicast request via a unicasttransmission, the request handler 2220 may determine that the data isintended for distribution to the computing device 2200. In anotherexample, the request handler 2220 may determine that the data isintended for distribution to other devices based on determining that thedata is received via a multicast transmission, which intended fordistribution to other devices connected to a network. Upon determiningthat data is to be communicated to a device, the request handler 2220may instruct the communication module 2216 to transmit the receiveddata. In some embodiments, the request handler 2220 may determinewhether to transmit the request based on a determination by theinformation manager 2214 as to whether the information 2232 stored inthe cache includes information to satisfy the request. In someembodiments, the received data may be transmitted after being processedby the information manager 2214. The information manager 2214 isdescribed further below.

In some embodiments, the request handler 2220 may determine whether torequest that data be transmitted to a device. The request handler 2220may determine whether to transmit data to a device, e.g., the device2302 or the device 2306, based on whether the information 2232 stored inthe cache 2230 includes updates to be shared with other devices. Anindication whether the information 2232 includes updates may bedetermined by the information manager 2214. Upon determining that theinformation 2232 includes updates, the request handler 2220 may instructthe communication module 2216 to transmit data including the updates.Updates may be transmitted via a unicast transmission or a multicasttransmission. In some embodiments, the request handler 2220 may requestfor updates to the information 2232 to be transmitted to other devicesupon determining that the information 2232 has been updated. In someembodiments, the request handler 2220 may transmit updated informationstored in the cache 2230 based on whether a resource threshold issatisfied. As explained further below, the resource determiner 2222 maydetermine whether the resource threshold is satisfied.

In some embodiments, the request handler 2220 may determine whether arequest for updates to the information 2232 stored on the cache 2230should be transmitted to other devices. The information manager 2214 mayprovide an indication as to whether to update the information 2232. Insome embodiments, the request handler 2220 may instruct thecommunication module 2216 to send a request to other devices to obtainupdates to the information 2232, if any. Such a request may includeversion information indicating a current version of the information 2232stored in the cache. The request to other devices may be sentperiodically or intermittently. The request may be sent before anotherdevice requests for updated information. In some embodiments, therequest for updates may be sent based on determining whether a resourcethreshold is satisfied. In some embodiments, the request handler 2220may instruct for a request for updates to the information 2232 to betransmitted based on receiving a request for updates from anotherdevice.

The information manager 2214 may manage storage of the information 2232in the cache 2230. The information manager 2214 may determine whether tostore information obtained from received data (e.g., data received froma transmission received using the communication module 2216) in thecache 2230. The information manager 2214 may process the received datato identify one or more types of information, e.g., criteria related tooperation of a device or device information. In some embodiments, thereceived data may indicate the types of information in the receiveddata.

In some embodiments, the version manager 2212 may process theinformation in the received data to determine a version of theinformation. The version may be determined from the received data or maybe determined by comparing the information from the received data withthe information 2232 on the cache 2230. The differences in theinformation may be used to determine a version of the information. Insome embodiments, received data may include version informationindicating a version corresponding to the different types ofinformation. In some embodiments, the received data may include amodifier that indicates changes information associated with devices. Forexample, the received data may include a modifier of a rule indicatingchanges to a previous version of a rule.

In some embodiments, the information manager 2214 may identify updatedinformation in data received from a device. The updated information mayinclude updates to the information 2232 stored on the cache 2230. Theinformation manager 2214 may update or modify the information 2232stored on the cache 2230 with the updated information. In someembodiments, the information manager 2214 may determine a differencebetween the updated information and the information 2232 stored on thecache 2230. The difference may be determined based on a version of theinformation determined for the updated information. The difference maybe determined based on a comparison of the updated information and theinformation 2232. In some embodiments, the comparison may be performedfor the type of information included in the updated information. Uponidentifying a difference between the updated information and theinformation 2232, the information manager 2214 may update the cache 2230with the updated information based on the identified difference. In someembodiments, updating the cache 2230 may include merging the information2232 with the updated information. For example, the updated informationmay include changes or modifications to the information 2232, such as achange in a criterion related to operation of a device. In suchinstances, the information 2232 may be modified based on themodification indicated by the updated information.

The rules manager 2218 may manage rules based on one or more criterion2240 stored on the cache 2230. In some embodiments, the rules manager2218 may determine when the computing device 2200 is to perform anoperation based on a criterion 2240. The operation may includeinstructing the communication module 2216 to transmit data with arequest to another device to perform an operation based on the criterion2240. In some embodiments, the rules manager 2218 may determine for theinformation manager 2214, differences between updated information andthe criteria 2240 stored on the cache 2230. The rules manager 2218 mayperform processing to merge the updated information with the criteria2240 to produce an updated criteria for storage in the cache 2230.

As mentioned earlier, the resource determiner 2222 may determineavailability and/or capabilities of computing resources (e.g.,processor(s), storage, cache, or the like) for the computing device2200. In some embodiments, the resource determiner 2222 may determinewhether a computing resource satisfies a resource threshold. Forexample, a resource threshold may be a particular amount of processingavailability. The resource threshold may be specified by informationstored in the cache 2230. The resource threshold may be configurable. Asexplained above, the resource determiner 2222 may provide informationindicating whether a resource threshold is satisfied. The resourcethreshold may be used to determine whether to request other devices forupdated information. For example, the computing device 2200 may requestfor updated information during greater processor availability. Otheroperations described as being performed by the computing device may beperformed based on satisfaction of a resource threshold.

FIGS. 23 and 24 illustrate sequence diagrams of processes of cachinginformation associated with network devices in accordance with someembodiments. Specifically, each of the sequence diagrams show an exampleof at least one scenario in which a computing device (e.g., thecomputing device 2200) may be implemented to cache informationassociated with network devices to improve responsiveness for operationof a device (e.g., a network device) connected to a network.

Now turning to FIG. 23, a sequence diagram of a process 2300 of cachinginformation associated with network devices is shown in accordance withsome embodiments. The sequence of steps shown for the process 2300 maybe implemented by a combination of a device 2302, a computing device2304, and a device 2306. The device 2302 can include a network device(e.g., the network device 302, the network device 304, or the networkdevice 306) or an access device (e.g., the access device 108). Thedevice 2306 can include a network device (e.g., the network device 302,the network device 304, or the network device 306) or an access device(e.g., the access device 108). The device 2302, the computing device2304, and the device 2306 may be communicatively connected through anetwork (e.g., the network 100 or the network 300). In some embodiments,the computing device 2304 may include a network device that is differentfrom other network devices connected to the network. For example, thecomputing device 2304 may include a network device, such as a router ora gateway (e.g., the gateway 110). In another example, the computingdevice 2304 may include any one of the network devices 302-308 that isdistinct from other devices (e.g., the device 2302 and/or the device2306) connected to a network. The device 2200 of FIG. 22 can be includedor implemented in any of the devices 2302-2306. In the example shown inFIG. 23, the computing device 2200 may be included or implemented in thecomputing device 2304. The network devices 2302-2306 may be incommunication with each other via a network (e.g., the network 300).

The process 2300 illustrated in FIG. 23 enables the computing device2304 to cache information associated with network devices connected to anetwork. For example, the information to be cached may be associatedwith the device 2302, the computing device 2304, the device 2306, orcombinations thereof.

By operating within a communication path of devices (e.g., the device2302 and the device 2304), the computing device 2304 may receiveinformation (e.g., a rule or device information) associated with otherdevices (e.g., a network device) connected to a network. The informationmay include current information associated with devices. The informationmay be acquired during communication between the devices 2302, 2306,which may be facilitated via the computing device 2304. The computingdevice 2304 can intercept communications between devices (e.g., thedevice 2302 and the device 2306) to cache information (e.g., a rule ordevice information) communicated between these devices. The cachedinformation can be provided to devices, whether requested or not, toimprove each of their response time for operation using currentinformation associated with devices (e.g., a rule or deviceinformation). An improvement in response time for obtaining currentinformation may enable a device, e.g., the device 2302, to improve itsown responsiveness for operation. For example, the device 2302 mayafford a faster response time for presenting the current information toits user and/or may operate faster by receiving a faster response withthe current information, which may be used by the device 2302 for itsoperation. The device 2304 may provide cached information to satisfy arequest from the device 2302 by intercept the request for currentinformation directed to devices on a network.

The process 2300 may begin by the device 2306 communicating with thedevice 2304. The process 900 may include the device 904 receiving a datatransmission 2310 from the device 2306. The data transmission 2310 maybe facilitated through use of a network. The data transmission mayinclude information associated with devices connected to the network.The information may include current information associated with devices.The information may include a criterion related to operation of adevice, device information, status information, other information storedby a device connected to the network, or combinations thereof. Theinformation may be distributed for synchronization of informationbetween the network devices connected to the network. Alternatively oradditionally, the data transmission 2310 may be performed forinformation discovered and/or created at the device 2306. For example,the information may be received upon registration of the device 2306 tothe network.

In some embodiments, the data transmission 2310 may occur in response toreceiving a request 2308 transmitted by the computing device 2304 to oneor more devices, e.g., the device 2306, connected to the network. Therequest 2308 may be sent periodically or intermittently to obtaincurrent information (e.g., criteria related to operation of a device ordevice information) stored by devices connected on a network, such asthe device 2306. For example, the computing device 2304 may send therequest 2308 to obtain updated information associated with devices,e.g., the device 2302 or the device 2306. The computing device 2304 maystore information (e.g., device information) about devices connected tothe network. The computing device 2304 may use device information storedin its cache to send the request 2308. In one example, the request 2308may be a unicast request sent directly to the device 2306. In anotherexample, the request 2308 may be a multicast request transmitted to thenetwork for delivery of the request 2308 to devices on the network. Insome embodiments, the computing device 2304 may transmit the request2308 to obtain information before other devices, e.g., the device 2302,request current information from other devices, such as the device 2306.

In certain embodiments, the computing device 2304 may include orimplement a network device (e.g., the gateway 110) that can receive datatransmissions from a network device, such as the network device 2306.The data transmission 2310 may be initiated by the device 2306 fordistribution to a network to which network devices (e.g., the device2302) are connected. In some embodiments, the computing device 2304 maybe a gateway that receives data transmissions (e.g., the datatransmission 2310) for communication to the network. Alternatively oradditionally, the computing device 2304 may be a network device thatreceives data transmissions, e.g., the data transmission 2310, fromother devices on the network, e.g., the device 2306.

In some embodiments, the device 2306 may be an access device, e.g., theaccess device 108. In such embodiments, the data transmission 2310 mayoccur upon creation or modification of information by the device 2306.Information such as rules may be created at the device 2306 through aninterface presented by the device 2306. The new rules may becommunicated to the network for distribution to devices, e.g., thedevice 2302 and the computing device 2304.

The process 2300 may include the computing device 2304 performing one ormore operations 2312 using information received from the datatransmission 2312. One operation 2312 may include storing informationreceived via the data transmission 2310. For example, the computingdevice 2304 may store information included in the data transmission 2310in a cache of the computing device 2304. The information received viathe data transmission 2310 may include updates to information alreadystored on the cache of the computing device 2304. In such instances, theupdated information may be stored on the cache. In some embodiments, theinformation included in the data transmission 2310 may includeinformation that is already existing on the computing device 2304.Another operation 2312 may include determining differences between theinformation stored on the cache and the information received via thedata transmission 2310. The differences, if any, may be stored orupdated to the information on the cache. In some embodiments, theinformation included in the data transmission 2310 may include anidentifier or a modifier indicating a version of the information and/orindicating the updates to the information stored on the cache.

In some embodiments, the device 2306 may transmit data, e.g., the datatransmission 2310, periodically or intermittently. In some embodiments,the data transmission 2310 may occur asynchronously from the requests,e.g., the request 2308, by the computing device 2304.

The process 2300 may include the device 2302 sending a request 2314 tothe computing device 2304. The request 2314 may be sent to obtaininformation (e.g., current information) associated with devices on thenetwork. The information requested may include a criterion related tooperation of one or more devices on the network (e.g., a rule), deviceinformation, status information, other information related to operationof a device on a network, or combinations thereof.

In some embodiments, the device 2302 may send the request 2314 directlyto the computing device 2304. The request 2314 may be sent when thedevice 2302 first registers with the network connected to the devices2302-2306 or later when the device 2302 requests for the information tosynchronize its locally stored information. For example, when thecomputing device 2304 is implemented as a gateway (e.g., the gateway110), the request 2314 may be a multicast request to the network forcurrent information associated with devices. In another example, thedevice 2302 may send the request 2314 as a unicast request directly tothe computing device 2304 for current information when the computingdevice 2304 is operating as a network device. In this example, therequest 2314 may be for current information, such as a criterion relatedto operation of a device on the network (e.g., a rule), deviceinformation, status information, other information related to operationof a device on a network, or combinations thereof. The request 2314 mayinclude type information indicating a type of information requested,such as a criterion related to operation of a device or deviceinformation. The request 2314 may include version information (e.g., aversion identifier) that indicates a version of the informationcurrently stored by the device 2302. In some embodiments, the request2314 may include information currently stored on the device 2302. Suchinformation may be used by a recipient device to determine differencesfrom the information stored locally on the recipient device.

The process 2300 may include the computing device 2304 performing one ormore operations 2316. One operation 2316 may include detecting that arequest 2314 is to obtain information associated with a device connectedto the network. For example, the computing device 2304 may determinewhether the request 2314 is a multicast request to obtain informationassociated with a device connected to the network. Upon detecting thatthe request 2314 is for information associated with a device connectedto the network, the computing device 2304 may perform another operation2316. In some embodiments, the operation 2316 performed based on thedetected request may include searching the cache of the computing device2304 for stored information associated with a device connected to anetwork. By searching the cache for information that may satisfy thedetected request 2314, the computing device 2304 may be able to locateinformation satisfying the request 2314 without having to communicate,or forward, the request 2314 to other devices to retrieve informationsought by the device 2302. The computing device 2304 may include in itscache all or some of the information that may satisfy the request 2314.The computing device 2304 can locate information satisfying the request2314 without having to request other devices (e.g., the device 2306) forinformation. By doing so, the computing device 2304 can provide a fasterresponse to the request 2314 from the device 2302. The device 2302 maynot have to wait as long to receive requested information. The fasterresponse may translate to improved responsiveness at the device 2302 foruse of the information. The device 2302 may be able to reduce responsetime for presenting or using the current information associated withdevices.

In some embodiments, the cache of the computing device 2304 may besearched using information (e.g., version information or typeinformation). For example, the cache may be searched to identifyinformation corresponding to the type information. In another example,the computing device 2304 may search the cache to identify informationhaving a version that is more current than the version indicated by theversion information received in the request 2314. The process 2300 mayinclude the computing device 2304 determining whether informationsatisfying the request has been located in the cache. In someembodiments, the cache may not store all the current information, whichmay satisfy the request. In such cases, the computing device 2304 maysend a request to other devices (e.g., the device 2306) for informationthat is not located on the cache. In some embodiments, the operation2316 may include determining a different between information stored onthe cache of the computing device 2304 and the information included inthe request 2314.

In some embodiments, the process 2300 may include the computing device2304 sending data to the device 2302 via a data transmission 2318. Thetransmitted data may include information located in the cache insatisfaction of the request 2314. By sending the information retrievedfrom the cache of the computing device 2304, the device 2302 may receiveall or some information responsive to the request 2314 faster. In someembodiments, the computing device 2304 may not have to send the request2314 to other devices on the network. The communication of a request toanother device (e.g., the device 2306) may incur additional time forretrieval of current information. Thus, by avoiding such communicationwhen information is located on the cache of the computing device 2304, aresponse time can be reduced for providing current information.

In some embodiments, the process 2300 may include the computing device2304 sending a request 2320 to one or more other devices, e.g., thedevice 2306, connected to the network. The request 2320 may be sent to adevice obtain information which can satisfy the request 2314. Suchinformation that may be retrieved from the device 2306 may beinformation that is not stored on the cache of the computing device2304. The request 2320 may be sent to multiple devices on the network,e.g., all devices on the network. For example, the request 2320 may besent as a unicast request directed to each individual device (e.g., thedevice 2306) connected to the network. In another example, the request2320 may be sent as a multicast request to be distributed to devices(e.g., the device 2306) connected to the network. In some embodiments,the request 2320 may be based on the request 2314. For example, when therequest 2314 is a multicast request, the computing device 2304 may sendthe request 2320 based on the request 2314 to devices on the network.

The request 2320 may include one or more criterion for requestinginformation from the device 2306. In some embodiments, the request 2320may include information included in the request 2314. For example, therequest 2320 may include type information and/or device information toindicate information sought by the computing device 2304. In someembodiments, request 2320 may indicate information that is sought, whichmay not have been stored locally on the cache of the computing device2304.

In some embodiments, to improve response time for providing informationthat satisfies the request 2314, the process 2300 may include thecomputing device 2304 transmitting data via the data transmission 2318concurrently with sending the request 2320. By doing so, a response timemay be reduced for retrieving information in satisfaction of the request2314. In some embodiments, the device 2302 may be able to update and/oruse the data received via the data transmission 2318 before information,if any, is retrieved from the device 2306 in response to the request2320.

The process 2300 may include the computing device 2304 receiving datavia a data transmission 2322 from one or more devices, e.g., the device2306, connected to the network. The data transmitted for the datatransmission 2322 may include information that satisfies the request2320. The process 2300 may include the computing device 2304 receivingthe data transmitted via the data transmission 2322.

The process 2300 may further include the computing device 2304performing one or more operations in response to the data received viathe data transmission 2322. The computing device 2304 may transmit datavia a data transmission 2326. The transmitted data may include the datareceived via the data transmission 2322 from the device 2306. In someembodiments, the computing device 2304 may combine data stored locallyon the computing device 2304 with data received from the device 2306.The data transmitted via the data transmission 2326 may include thecombined data. The received data may be merged with cached information.The computing device 2304 may determine differences between the cachedinformation and the received data and may store the differences. Versioninformation in the cached information and the received data may be usedto determine the differences.

By implementing a computing device that can communicate with devices ona network, the computing device can store information (e.g., a criterionrelated to operation of a device, device information, etc.) on thecomputing device for quick retrieval when a device requests for updatedinformation. In doing so, additional communication with devices on thenetwork can be reduced for retrieving the information associated with adevice. Further, bandwidth consumption in a network can be reduced basedon a reduction in communication between devices to retrieve currentinformation associated with a device.

Now turning to FIG. 24, a sequence diagram of a process 2400 of cachinginformation associated with network devices is shown in accordance withsome embodiments. The sequence of steps shown for the process 2400 maybe implemented by a combination of an access device 2402, a cloudnetwork 2404, a computing device 2406, and a device 2408. The accessdevice 2402 may include or implement the access device 108. The device2408 can include a network device (e.g., the network device 302, thenetwork device 304, or the network device 306) or an access device(e.g., the access device 108). The access device 2402, the cloud network2406, the computing device 2408, and the device 2408 may becommunicatively connected through a network (e.g., the network 100 orthe network 300). In some embodiments, the computing device 2406 mayinclude a network device that is different from other network devicesconnected to the network. For example, the computing device 2406 mayinclude a network device, such as a router or a gateway (e.g., thegateway 110). In another example, the computing device 2406 may includeany one of the network devices 302-308 that is distinct from otherdevices (e.g., the device 2402 and/or the device 2406) connected to anetwork. The device 2200 of FIG. 22 can be included or implemented inany of the devices 2402-2408. In the example shown in FIG. 24, thecomputing device 2200 may be included or implemented in the computingdevice 2406. In some embodiments, the cloud network 2404 may include orimplement the cloud network 114.

The process 2400 illustrated in FIG. 24 enables the computing device2406 to cache information associated with network devices connected to anetwork (e.g., the network 300). Specifically, the process 2400 showshow a computing device, e.g., the computing device 2406 storinginformation (e.g., the information 2232) can improve responsiveness foran access device (e.g., the access device 2402) to operate and/orpresent information associated with devices (e.g., device information ora criterion related to operation of a device) when the access device iscontrolled remotely via a cloud network, e.g., the cloud network 2404.In some embodiments, the computing device 2406 can store informationassociated with network devices as it is received from the cloud network2404 or other devices, e.g., the device 2408. In some embodiments, thecomputing device 2406 can communicate with network devices to obtaininformation including updates to existing information stored in a cacheof the computing device 2406. As described above with reference to FIG.23, by maintaining communication with a device (e.g., the device 2408),the computing device 2406 may receive information (e.g., rules or deviceinformation) associated with devices connected to a network. Thecomputing device 2406 may maintain current information associated withdevices such that the computing device 2406 may provide information toother devices to improve their responsiveness for presenting and/oroperating using the information.

The process 2400 may include the access device 2402 transmitting data tothe cloud network 2404 via a data transmission 2416. A request forinformation associated with devices, e.g., the device 2408) may be sentvia the data transmission 2416. For example, a request may be sent viathe data transmission 2416 to obtain device information and/or acriterion related to operation of a device to present at the accessdevice 2402, such as the access device shown in FIGS. 20 and 21. In someembodiments, the data transmission 2416 may transmit current informationassociated with devices on the network. For example, the access device2402 may communicate updates to device information and/or a criterionrelated to operation of a device to the network to be provided to otherdevices, e.g., the computing device 2406 and/or the device 2408.

The process 2400 may include the cloud network performing one or moreoperations 2418 in response to data transmitted via the datatransmission 2416. For example, the cloud network 2404 may storeinformation received via the data transmission 2416. The storedinformation may be retrieved in response to a request for informationfrom the access device 2402. The cloud network 2404 may search its localstorage for cached information that may satisfy a request forinformation received via the data transmission 2416. The cachedinformation may include information such the type of information storedin the cache 2230 of the computing device 2200. In some embodiments, theprocess 2400 may include the cloud network 2404 communicating with theaccess device 2402 via a data transmission 2414. The cloud network 2404may send via the data transmission 2414 information responsive to arequest received via the data transmission 2416.

In some embodiments, the cloud network 2404 may communicate with thecomputing device 2406 via a data transmission 2420. The cloud network2404 may send, via the data transmission 2420, a data received from theaccess device 2402. The data received from the access device 2402 mayinclude the data received by the cloud network 2404 via the datatransmission 2416. In some embodiments, the cloud network 2404 may senda new request via the data transmission 2420 to obtain information notretrieved from local storage of the cloud network 2404.

The process 2400 may include the computing device 2406 communicatingwith one or more network devices, e.g., the device 2408, via a datatransmission 2424. The computing device may store information associatedwith devices on a network. The information may include a criterionrelated to operation of a device, device information, or statusinformation. The computing device 2406 may send its cached informationto the device 2408 periodically or intermittently to enable the device2408 to remain synchronized with its information. In some embodiments,the computing device 2406 may send a request to the device 2408 forupdates to information (e.g., criteria related to operation of devicesand device information) stored on the computing device 2406. The requestmay include a version identifier to identify a version of theinformation stored on the cache of the computing device 2406. Therequest may be sent before, after, or concurrently with receiving thedata transmission 2420.

The process 2400 may include the device 2408 communicating with thecomputing device 2406 via a data transmission 2426. The device 2408 maytransmit via the data transmission 2426, information associated withdevices (e.g., a criterion related to operation of devices or deviceinformation). The information may be sent in response to a request viathe data transmission 2424. The information may include updates toinformation associated with devices and may include version informationindicating a version of the information that is sent. The device 2408may send updates to the information periodically or intermittently.

The process 2400 may include performing one or more operations 2422 tostore the data received via the data transmission 2426. One operationmay include storing information included in the data received from thedevice 2408. In some embodiments, the computing device 2406 maydetermine differences between information in the data received via thedata transmission 2426 and cached information on the computing device2406. The computing device 2406 may store information received from thedevice 2408 that is different from the data stored in its cache.

The process 2400 may include the computing device 2406 performing one ormore operations 2428 in response to a request received via the datatransmission 2420. In some embodiments where the data transmission 2420includes information associated with devices, one operation may includethe computing device 2406 storing such information received from thecloud network 2404. The computing device 2406 may determine differencesbetween information received via the data transmission and cachedinformation on the computing device 2406. The computing device 2406 maystore the information received from the cloud network 2404 that isdifferent from the cached information. Another operation may include thecomputing device. Another operation may include detecting that a request(e.g., a multicast request) is received, via the data transmission 2420,for information associated with devices. Upon detecting a request forinformation, the computing device 2406 may perform another operation forsearching its cache for stored information satisfying the request. Bystoring information associated with devices in the cache of thecomputing device 2406, the computing device 2406 may have all or someinformation of the request. As such, the computing device 2406 maysearch its cache for information requested from the cloud network 2404and may retrieve information located in the cache. By locating suchinformation from the cache, the computing device 2406 may provideinformation to the access device 2402 faster, without having to requestthe information from other devices, e.g., the device 2408. By providingthe information faster, the access device 2402 may improve itsresponsiveness based on receiving the information. For example, theaccess device 2402 may present updated information about rules and/ordevice information faster in a graphical interface such as those shownin FIGS. 20 and 21.

The process 2400 may include the computing device 2406 sending theinformation retrieved from the cache to the cloud network 2404 via adata transmission 2432. In some embodiments, although not shown, thecomputing device 2406 may send the information directly to the accessdevice 2402 if the access device 2402 becomes connected to a network towhich the computing device is connected.

The process 2400 may include the computing device 2406 sending a requestto the device 2408 via a data transmission 2430 to obtain informationassociated with the devices. Information that is not located in thecache may be requested from the device 2408 via the data transmission2430. In some embodiments, the data transmission 2430 and the datatransmission 2432 may occur concurrently to improve response time forretrieving information requested via the data transmission 2420.

The process 2400 may include the cloud network 2404 sending, via a datatransmission 2442, the information received from the computing device2406 via the data transmission 2432. The access device 2402 may controloperation of devices and/or presenting information based on theinformation received via the data transmission 2442. In someembodiments, the cloud network 2404 may perform operations 2444, such asstoring in its local cache the information received via the datatransmission 2432.

The process 2400 may include the device 2408 sending, to the computingdevice 2406 via a data transmission 2434, the information associatedwith devices (e.g., a criterion related to operation of devices ordevice information) requested via the data transmission 2430. Thecomputing device 2406 may perform one or more operations in response tothe data received via the data transmission 2434. The computing device2406 may store the received data. In some embodiments, the computingdevice 2406 may determine differences between cached information and thereceived data. Such differences may be determined using versioninformation and/or type information determined from the data. Thecomputing device 2406 may send the received data to the cloud network2404 via a data transmission 2438. In some embodiments, the computingdevice 2406 may send updates or differences from the cached informationvia the data transmission 2438.

The process 2400 may include the cloud network 2404 performing one ormore operations 2440 in response to receiving data via the datatransmission 2438. One operation 2440 may include storing the datareceived via the data transmission 2438. In some embodiments, the cloudnetwork 2404 may determine differences between previously storedinformation associated with devices and information associated withdevices included in the data received via the data transmission 2438.The cloud network 2404 may store the differences in local storage.Another operation 2440 may include the cloud network 2404 sending theinformation to the access device via a data transmission 2446. The cloudnetwork 2404 may send the information received via the data transmission2438 to the access device 2402. In some embodiments, the cloud network2404 may send the differences determined based on the storedinformation.

Upon receiving the information via the data transmission 2446, theaccess device 2402 may perform one or more operations 2448 using thereceived information. One operation 2448 may include displaying all orsome of the information received via the data transmission in aninterface (e.g., the interface 2005 or the interface 2105) of the accessdevice 2402. Another operation 2448 may include controlling operation ofdevices based on the received information.

Now turning to FIG. 25, a flowchart is shown illustrating a process 2500for caching information in according with some embodiments.Specifically, the process 2500 provides techniques for storinginformation (e.g., the information 832), such as criteria (e.g.,criteria related to operation of devices), device information, statusinformation, icons, meta-information, or interfaces) received fromnetwork devices (e.g., any of the network devices 302-308) and/or accessdevices (e.g., the access device 108) on a network. By implementing theprocess 2500, information received for communication from one device toanother via a device (e.g., a router device) can be stored to improveresponse time for retrieving such information. In some embodiments,process 2500 may be performed by a computing device, e.g., the computingdevice 800. Requests from a network device to other devices (e.g.,network devices 302-308 and/or access devices 108) for data, such asrules, device information, status information, or other data, may beretrieved from local cache by the process 2500 and provided back to arequesting device. By intercepting requests at a computing device (e.g.,a router device), the process 1200 can reduce communication with one ormore devices to obtain information associated with devices. A computingdevice in communication with devices on a network can store informationas it is received from other devices. The computing device can determinedifferences to maintain current information. By maintaining currentinformation, the computing device can provide current informationrequested by a device, thereby improving a response time for retrievingthe information.

Process 2500 is illustrated as a logical flow diagram, the operation ofwhich represents operations that may be implemented in hardware,computer instructions, or a combination thereof. In the context ofcomputer instructions, the operations represent computer-executableinstructions stored on one or more computer-readable storage media that,when executed by one or more processors, perform the recited operations.Generally, computer-executable instructions include routines, programs,applications, objects, components, data structures, and the like thatperform particular functions or implement particular data types. Theorder in which the operations are described is not intended to beconstrued as a limitation, and any number of the described operationsmay be combined in any order and/or in parallel to implement theprocesses.

Additionally, the process 2500 may be performed under the control of oneor more computer systems configured with executable instructions and maybe implemented as code (e.g., executable instructions, one or morecomputer programs, or one or more applications) executing collectivelyon one or more processors, by hardware, or combinations thereof. Asnoted above, the code may be stored on a computer-readable storagemedium, for example, in the form of a computer program comprising aplurality of instructions executable by one or more processors. Thecomputer-readable storage medium may be non-transitory.

Beginning at block 2502, the process 2500 may include receivinginformation associated with a device (e.g., the network device 302, thenetwork device 304, the network device 306, or the network device 308)connected to a network (e.g., the network 300). For example, the process2500 may receive from a network device (e.g., the network device 302)device information about network devices connected to a network. In someembodiments, the received information may include device informationidentifying one or more devices connected to a network. The deviceinformation may include device identifiers, each device identifiercorresponding to a distinct device connected to a network. In anotherillustrative example, the process 2500 may include receiving, from anaccess device, a criterion related to operation of one or more devices(e.g., the network device 302, the network device 304, the networkdevice 306, or the network device 308) connected to a network. Thecriterion may include rules for performing actions, which indicatesettings and/or attributes related to operation of a device. Forexample, the criterion may include an operation criterion such as valueor a range of values for a setting and/or an attribute related tocontrolling operation of a device. In another example, the criterion mayinclude an instruction criterion that indicates an instruction foroperating a device. An instruction may include a condition by which tocontrol operation of a device. The condition may be based on time, date,weather, location, an event, or the like, when a device is to perform anoperation.

At block 2504, the process 2500 may include storing the information(e.g., the information received at block 2502) on a cache of a computingdevice. In some embodiments, the computing device which includes thecache may be the same computing device that performs the process 2500.

At block 2506, the process 2500 may include receiving updatedinformation associated with the device. For example, the process 2500may receive updated information associated with a device from a deviceconnected to a network. The device may include a network device (e.g.,the network device 302, the network device 304, the network device 306,or the network device 308) or an access device (e.g., the device 108).The updated information may be received from a network device that isdifferent from a network device from which the stored information wasreceived. In some embodiments, the updated information may include a newcriterion (e.g., a new rule) for operation of a device (e.g., thenetwork device 302, the network device 304, the network device 306, orthe network device 308). The updated information may be received from anetwork device that is associated with the updated information. In someembodiments, updates to the information stored on the cache may bereceived in parts, each from a different device connected to a network.In some embodiments, the updated information may be received from adevice (e.g., the access device 108) that is different from the deviceassociated with the updated information. In some embodiments, theupdated information may include device information identifying newdevices connected to a network. The updated information may includeupdates to the information stored in the cache. For example, the updatedinformation may include updates to a criterion (e.g., a rule) related tooperation of a device. The updates to the criterion may include anidentifier that indicates the modification to the criterion (e.g., arule modifier). The updated information may include informationindicating a version (e.g., a version identifier) corresponding to theupdated information. In some embodiments, the updated information mayinclude updated device information, which may indicate updates to thedevice information stored on the cache. The updated device informationmay indicate updates to the devices connected to a network. Such updatesmay indicate an update of the devices connected to the network. Theupdates may be used to determine devices that are no longer connected tothe network.

At block 2508, the process 2500 may include modifying the informationstored on the cache using the updated information. A computing devicethat implements the process 2500 may be the computing device includingcache that is modified with respect to the stored information. In someembodiments, the process 2500 may modify information stored on the cacheby updating the information stored on the cache based on the updatesreceived in the updated information. The process 2500 may determinedifferences between the stored information and the updates. The process2500 may modify the information stored on the cache based on thedetermined differences. In some embodiments, the process 2500 may modifythe information stored on the cache by replacing the information withthe updated information. In some embodiments, the information stored onthe cache may be modified with the updated information based on versioninformation included in the updated information. When the updatedinformation includes new information not stored on the cache, theinformation stored on the cache may be modified to include the newinformation.

At block 2510, the process 2500 may include detecting a request forcurrent information associated with a device. For example, the process2500 may detect a request from an access device (e.g., the access device108) for current information associated with a network device (e.g., thenetwork device 302, the network device 304, the network device 306, orthe network device 308). In this example, the request from the accessdevice may be for current information associated with a device. Suchinformation may be used, for example, to display a graphical interfacethat presents information about devices (e.g., a device list) connectedto a network. The request for current information associated withdevices connected to a network may be transmitted via a multicastrequest. The process 2500 may detect a request for current information,such as when a multicast request is directed to devices connected to anetwork. In some embodiments, the request for current information mayinclude version information indicating a version of information storedby the device that sends the request. The version information may beused to detect that a request is for current information associated witha device.

At block 2512, the process 2500 may include retrieving the currentinformation from the information stored on the cache. The currentinformation may be retrieved from cache of a computing deviceimplementing the process 2500. The current information that is retrievedmay include all information stored on the cache. In some embodiments,the process 2500 may determine information to retrieve based oninformation indicated in the detected request at block 2510. The requestmay indicate a type of information (e.g., a criterion related tooperation of devices or device information) that is requested. In someembodiments, version information included in the detected request may beused to retrieve the current information. For example, based on versioninformation, the process 2500 may determine information stored on thecache that is different (e.g., new or updated) that information storedon a device that sent the detected request. The version indicated in thedetected request may be compared to version information associated withinformation stored on the cache. Information stored on the cache may bedifferent when a version does not match a version indicated by thedetected request. Information that corresponds to a different versionmay be identified as current information to be retrieved by the process2500.

At block 2514, the process 2500 may include transmitting the retrievedcurrent information to the network. For example, the process 2500 maytransmit the retrieved current information to a network on which one ormore network devices may be connected. By transmitting the retrievedcurrent information to the network, other network devices on thenetwork, such as those which did not send the detected request, mayreceive updated information. The network devices may be able todetermine whether the current information contains updates toinformation stored on each of the network devices individually. In someembodiments, the retrieved information may be transmitted to thenetwork, but directed to a device that sent the detected request. Insome embodiments, the retrieved current information may be transmittedwith information indicating the type of information included in thecurrent information. The retrieved current information may betransmitted with version information to indicate a version correspondingto the current information. For example, the retrieved currentinformation may include version information (e.g., a version identifier)corresponding to one or more criterion related to operation of devices.In another example, the retrieved current information may includeversion information for device information. The information indicatingthe type of information included in the retrieved current informationmay enable a device (e.g., an access device or a network device) todetermine information to update locally on the device.

It will be appreciated that process 2500 is illustrative and thatvariations and modifications are possible. Steps described as sequentialmay be executed in parallel, order of steps may be varied, and steps maybe modified, combined, added or omitted. In some embodiments, steps ofthe process 2500 may be performed during different time periods or theblocks may correspond to steps performed in a different order thanshown.

In some embodiments, steps of the process 2500 may be performed duringdifferent time periods or the blocks may correspond to steps performedin a different order than shown.

In some embodiments, the blocks 2502 and 2504 may be performed during aninitialization time period 2550. During the initialization time period2550, a computing device that implements the process 2500 may receiveinformation associated with devices connected to a network. Suchinformation may be received, for example, from an access device that iscommunicating the information upon configuration by a user. In anotherexample, the information associated with a device connected to a networkmay be received via a communication from one network device to anothernetwork device via a computing device connected to the same network. Theblock 2502 and 2504 may be implemented for the process 2500 each timenew information is received. For example, new information may bereceived when a new device is discovered on the network 2500.

In some embodiments, the blocks 2506 and 2508 may be performed during anupdate time period 2560 during which a computing device that performsthe process 2500 may receive updates to information stored on thecomputing device. The update time period 2560 may occur immediatelyafter the initialization time period 2550 or may occur after a timeperiod occurring after the initialization time period 2550. The blocks2506 and 2508 may be implemented when information is to be stored on thecache is updated. For example, updates to different types of informationmay be received during different update time periods 2560. The updatedinformation may be received from a network device via the network. Theupdated information may be received from an access device via thenetwork. The updated information may be generated based on an update tosuch information at the access device.

In some embodiments, the blocks 2510-2514 may be performed during arequest time period 270. The request time period 2570 may occur afterthe initialization time period 2550. In some embodiments, the requesttime period 2570 may occur after the update time period 2560. The blocks2510-2514 may be implemented for the process 2500 when a request isreceived. During the request time period, the process 2500 may detect arequest for current information associated with a device connected to anetwork. The request may be detected for different types of information,such as criteria related to operation of a device and deviceinformation. Requests for different types of information may be receivedat different times, each occurring during a different request timeperiod 2570.

FIG. 26 illustrates an embodiment of a process 2600 for providing avisual interface module for controlling a network device. As shown, theprocess 2600 may be performed by one or more computing devices, such asthe network device 102, a server associated with cloud network 114, orthe access device 108 described above with reference to FIG. 1. In someembodiments, the network device 102 is associated with a home automationnetwork, such as the local area network 100 described above with respectto FIG. 1. Process 2600 is illustrated as a data flow diagram, theoperation of which represents operations that can be implemented inhardware, computer instructions, or a combination thereof. Gateway 110is connected to cloud network 114, and allows network device 102 toconnect to the cloud network 114, the Internet, or other externalnetworks via gateway 110. In some embodiments, the network device 102may be a home automation network device that allows a user to access,monitor, control, and/or configure various appliances, such as atelevision, radio, light bulb, microwave, iron, fan, space heater,sensor, and/or the like. In some embodiments, the user can monitor andcontrol network devices by interacting with a visual interface renderedby the gateway 110 (i.e., a web page for gateway 110), a visualinterface rendered on display 2622 of the access device 108, or a visualinterface rendered by the network device 102.

In an embodiment, an application may be run on the access device 108.The application may cause the access device 108 to present a display2622 with a modular visual interface for each network device accessibleon the local area network 100. When the application is run on the accessdevice 108, the access device 108 can access a cache 2602.

The cache 2602 can be a local cache located in onboard storage of theaccess device 108. The cache 2602 can contain a known interface list2620 with records 2624, 2626 and 2628 including interface informationfor different, known types of network devices. As shown, each of records2624, 2626 and 2628 can include a device type, a unique interface moduleID, and controls information. The known interface list 2620 can includea record for each device known by the access device 108 to exist on thelocal area network 100. When the application is run on the access device108, the access device 108 can access the known interfaces 2620 in thecache 2602 to present the display 2622, which lists modular interfacesfor each network device on the local area network 100. In an example,the display 2622 can include a modular tile for each connected networkdevice having an interface in the known interface list 2620. Examples ofcommunications used to populate cache 2602 are described in thefollowing paragraphs.

The process 2600 can include utilizing communication 2606 to register avisual interface module for a network device 102 with a server of cloudnetwork 114. For simplicity, communication 2606 is shown as a directcommunication between network device 102 and cloud network 114. However,it is to be understood that in an embodiment, communication 2606 can besent from a manufacturer of network device 102 to cloud network 114. Inan additional or alternative embodiment, communication 2606 is sent fromthird party interface developer to cloud network 114. For example, athird party developer of a visual interface module for network device102 may initiate communication 2606 to cloud network 114. In the exampleof FIG. 26, communication 2606 includes registration information for thenetwork device 102. For example, communication 2606 can include a uniquedevice ID for network device 102. In some embodiments, the registrationinformation may identify one or more capabilities of network device 102.The registration information can include a unique identifier for thenetwork device, a default name of the network device, one or morecapabilities of the network device, and one or more discovery mechanismsfor the network device. In one example, communication 2606 can include aresource bundle corresponding to network device 102. The resource bundlecan be embodied as a structured folder structure whose contents defineall visual and interactive elements/areas in a tile. For example, aresource bundle can be a zip file sent from a device manufacturer or athird party developer that is submitted or uploaded to cloud network114. The resource bundle includes a unique device ID and files defininggraphical content of a visual interface module. The graphical contentcan include definitions of interactive elements/areas for the interfacemodule. The resource bundle can include templates defining interactivecontrol states for each of the interactive elements, languagetranslations for tile text, any menus for the tile, and graphicalcontent of the menus. For example, the resource bundle can definetemplates, text, and graphical content using a markup language, such asHTML5.

At 2606, the process 2600 includes transmitting an indication thatnetwork device 102 is associated with the network. For example, networkdevice 102 may transmit the indication to the server of the cloudnetwork 114. In some embodiments, transmitting may include transmittinga unique identifier (ID) for the network device 102. For example, thenetwork device 102 may send a communication to the server indicating aunique interface module ID for the network device 102. In suchembodiments, the server may then determine that a match between theunique interface module ID and a known interface exists. The cloudnetwork 114 can include a data store 2604 of known interfaces. Theaccess device 108 can download a visual interface module identified indata store 2604 from the cloud network 114, which can be used to rendera modular interface within display 2622. In an embodiment, data store2604 can be a tile database where each record in the database isuniquely identified by a tile ID.

Cloud network 114 can use the unique device ID to determine an interfacemodule for network device 102. As shown in FIG. 26, cloud network 114can access a data store 2604 of visual interface modules. A plurality ofuniquely identified interface modules can be stored in data store 2604.For example, each interface module in data store 2604 can be associatedwith a unique interface module ID. In an embodiment, data store 2604 isa database configured to store modular tiles for a plurality of networkdevices, with each of the stored modular tiles being identified by aunique tile ID. For instance, the network device 102 having a uniquedevice identifier may be matched with an existing interface module basedon comparing information received from the network device 102 withinformation stored in data store 2604. In cases where an existinginterface module for network device 102 is not found in data store 2604,cloud network 114 can use information in a resource bundle for thenetwork device 102 to generate an interface module, where the resourcebundle is provided as part of a registration process for a given networkdevice. The generated interface module can then be stored in data store2604 and assigned a unique interface module ID. In some embodiments,information in the resource bundle can be used to update an existinginterface module stored in data store 2604. After determining theinterface module for network device 102, cloud network 114 sendscommunication 2608 to network device 102 in order to provide a uniqueinterface module ID to the network device 102. In one embodiment,communication 2608 can include a unique tile ID corresponding to amodular tile for network device 102 that is stored in data store 2604.In some embodiments, communication 2608 includes a unique tile IDcorresponding to a modular tile defined for network device 102. Uponreceiving communication 2608 with the unique interface module ID (i.e.,a unique tile ID), the network device 102 can store the unique interfacemodule ID. In one embodiment, for example, the unique interface moduleID can be stored by an interface device 2601 of the network device 102that is configured to provide the interface module ID to an accessdevice or gateway. In an embodiment, the interface device 2601 isimplemented as a ‘smart module’ in hardware and firmware, such as, forexample, a system on a chip (SOC) integrated into the network device102.

The interface device 2601 can include flash memory and dynamic randomaccess memory (DRAM). The flash memory may be used to store instructionsor code relating to an operating system, one or more applications, andany firmware. The flash memory may include nonvolatile memory so thatany firmware or other program can be can updated. In the event theinterface device 2601 loses power, information stored in the flashmemory may be retained. The DRAM of the interface device 2601 may storevarious other types of information needed to run the interface device2601, such as all runtime instructions or code. The flash memory or DRAMor a combination thereof may include all instructions necessary tocommunicate with network device 102.

The process 2600 can include sending, from the access device 108,intra-network communication 2610 including a query, to the networkdevice 102. The query can be a request for information such as a queryfor capabilities, a request for an identity of the network device 102,and/or a request for a unique interface module ID. For example,communication 2610 can be sent from access device 108 to network device102 to query network device 102 about its identity. In response to thequery sent from access device 108, the process 2600 can includereceiving intra-network communication 2612 at the access device 108 withdevice information for the network device 102. According to anembodiment, in response to the query, the network device 102 can sendcommunication 2612 to inform the access device 108 of the identityand/or capabilities of the network device 102. For instance, in responseto receiving the query, the network device 102 may send communications2612 to the access device 108 with at least a unique interface moduleID. The process 2600 can include utilizing intra-network devicecommunications 2610 and 2612 as part of a discovery process for thenetwork device 102. For example, when the network device 102 isinitially connected to the network, it and access device 108 canautomatically exchange communications 2610 and 2612 to provide theaccess device 108 with information that can be used to determine abasic, default visual interface stored in cache 2602.

Within the context of a modular tile framework, embodiments candynamically render a functional user interface without having todownload the appropriate interface template from a remote server, suchas a server associated with the cloud network 114, in order to control anewly discovered network device. These embodiments can be used in caseswhere a connection to the Internet or the cloud network 114 isunavailable or unreliable, and immediate use of a newly discoverednetwork device is desired. In this case, an application on the accessdevice 108 or a stationary device such as gateway 110 could, based oncertain information received from the network device 102, dynamicallyrender a functional interface for immediate use. Such a functionalinterface may not be the ideal, visually optimized, interface that isdownloadable from the cloud network 114. However, such a functionalinterface will suffice until the application is able connect to theInternet and/or the cloud network 114 and subsequently download theappropriate and visually optimized interface module for the networkdevice 102.

In some embodiments, communication 2612 may be received when the networkdevice 102 is rebooted (e.g., powered on, reset or restored to defaultsettings, or the like). For example, when the network device 102 isrebooted, it may broadcast one or more messages on the local areanetwork 100 to discover whether there are any access devices in thelocal area network 100. For example, communication 2612 may be broadcastaccording to a UPnP protocol during a discovery process. The networkdevice 102 may receive communications 2610 from access device 108indicating that it is located within the local area network 100 andinterrogating network device 102 about its functionalities. That is,after receiving a broadcast message from network device 102, accessdevice 108 may then query network device 102 by sending communication2610 in order to receive the communication 2612 including informationabout the network device.

After receiving communication 2612, if the access device 108 can accessthe cloud network 114, it sends a communication 2614 to the cloudnetwork 114 as a request for an interface module for the network device102. Communication 2614 is described in further detail below. Otherwise,if the access device 108 cannot access the cloud network 114, the accessdevice 108 looks up the unique interface module ID received from thenetwork device 102 in cache 2602. As discussed above, cache 2602 can bea local cache stored on the access device 108. Basic properties forknown interfaces can be stored in cache 2602 as a device type andcontrols information. These basic properties can include, for example, adefault icon, a default name, and interactive elements or interactiveareas for controlling one or more primary functionalities of a networkdevice. The primary functionalities can include, for example, poweringthe network device on and off. The basic properties can also includecontrols information for secondary functionalities.

In some embodiments, when the access device is connected to the cloudnetwork 114, the access device 108 sends communication 2614 to query thecloud network 114 about network device 102. The communication 2614 caninclude at least the unique interface module ID for the network device102. At this point, the cloud network 114 can compare the uniqueinterface module ID of the network device 102 to known interface moduleIDs stored in data store 2604 in order to determine that there is amatch between the unique interface module ID sent with communication2614 and a known interface module. If the cloud network 114 finds aninterface module in its data store 2604, it transmits the interfacemodule to the access device 108 via communication 2616. For example, ifthe access device 108 is currently using a default interface module fornetwork device 102 that was determined based on exchangingcommunications 2610 and 2612, and then subsequently is able to connectto the cloud network 114, communications 2614 and 2616 between theaccess device 108 and the cloud network 114 can be used to obtain anupdated interface module for the network device 102.

Upon receiving communication 2616 from the cloud network 114, the accessdevice 108 populates a record in cache 2602 corresponding to the networkdevice 102 with device type and controls information received viacommunication 2616. That is, when the access device is remote from thelocal area network 100, it can exchange communications 2614 and 2616with the cloud network 114 to receive an interface module for a networkdevice. Information received via communications 2616 can be used topopulate records of cache 2602. Records in cache 2602 can be updatedusing modular interfaces received via communication 2616. In additionalor alternative embodiments, new records can be created in cache 2602when communication 2616 includes a modular interface for a newlydiscovered network device.

Records 2624, 2626, 2628 in cache 2602 store device types, uniqueinterface module IDs, and controls information for known networkdevices. The access device 108 uses the records in cache 2602 to rendervisual interfaces in the display 2622. For example, the display 2622 caninclude a navigable list of modular tiles corresponding to networkdevices in the local area network 100.

Display 2622 can also include an indicator representing a state ofnetwork device 102. In embodiments, communications 2612 and/or 2614 caninclude a last known state of the network device 102 and/or historicaldata associated with the network device 102. In one embodiment, suchstate information can be based on information received via communication2612 from the network device 102 when the access device 108 is connectedto the local area network 100. In this way, display 2622 of the accessdevice 108 can reflect a current state and historical data for thenetwork device 102 when the access device is not connected to the localarea network 100. In additional or alternative embodiments, the stateinformation can be based on information received via communication 2616from the cloud network 114 when the access device 108 is connected tothe cloud network 114. Using the state information, an interface moduleor tile for the network device 102 within display 2622 can indicate an‘on’ or ‘off’ state for the network device 102 when the network deviceis powered on or off.

FIG. 27A illustrates an example of a network 2700, according toembodiments of the present invention. Specifically, the network 2700 canbe a wireless local area network enabling an access device tocommunicate with network devices to control adjustment of attributesrelated to operation of the network devices. Network 2700 includesnetwork device 2702, network device 2704, network device 2706, andnetwork device 2708. The network 2700 also includes access device 108.In other words, the network 2700 may be substantially similar to thenetwork 100 except that access device 108 has been turned on near thenetwork 2700, to which it is associated, or has entered an area to whichthe network 2700 can reach.

When access device 108 can enter the network 2700 as shown in FIG. 27A,access device 108 may be authenticated based on the access device'sauthentication with the logical network or may otherwise commencecommunication with cloud network 114. Access device 108 may alsocommunicate notification of its presence or other information directlyto other network devices 2702-2708 within network 2700, as shown in FIG.27A by communication paths 2730. As noted, such communication mayinclude various communications protocols, such as Universal Plug andPlay (UPnP), Bluetooth®, Zigbee®, Ultra-Wideband (UWB), WiFi-Direct,WiFi, Bluetooth® Low Energy (BLE), sound frequencies, and/or the like.For example, access device 108 may communicate to all other devices innetwork 2700, including network device 2702, network device 2704,network device 2706, and network device 2708, information/data regardingits status. Such status data may include the fact that it is present andturned on, or other status data/information. At any time that networkdevices 2702, 2704, 2706 and 2708 recognize that access device 108 ispresent at network 2700, the network devices may communicate back toaccess device 108. For example, the network devices may send anacknowledgement (e.g., ACK signal) back to access device 108 to confirmthat they received the status data sent by access device 108. Thenetwork devices may also send their own status data to access device108.

While network devices 2702-2708 and access device 108 may each receivecommunication from other network devices around the network 2700,including the status of each of those network devices, network devices2702-2708 and/or access device 108 may be continuously scanning network2700 (including, for example, running discovery algorithms) to determinewhether any devices within the network have moved, turned on/off orotherwise added to or subtracted from the network 2700, or haveotherwise changed statuses.

Since network devices 2702-2708 and access device 108 may each receivecommunication from other devices around network 2700, including thestatus of each of those devices, each network device within network 2700may know the status of each other network device in the network 2700.For example, access device 108 or devices 2702-2708 may not be requiredto communicate with cloud network 114 in order to obtain one or more ofsuch statuses. Since cloud network 114 is an external network and may beremote from network 2700, communication between network devices withinthe network 2700 and cloud 114 may take more time than communicationbetween two devices within network 2700. For example, communicationbetween devices within network 2700 may take anywhere from 1 millisecondto 100 milliseconds, while communication between a device within network2700 and the cloud network 114 may take anywhere from 50 milliseconds to1 second or more). Furthermore, if a network device is retrievinginformation from cloud 114, the request must travel from the networkdevice to cloud network 114, and then the information must travel backfrom cloud network 114 to the network device. This process may doublethe latency caused by retrieving information with cloud 114. Therefore,devices within the network 2700 may choose to send and receive/retrievestatuses directly with other devices within the network 2700 instead ofcommunicating such information via cloud network 114. When a networkdevice receives status data from another network device on the device'slocal area network 2700, it may store that status data so that it mayretrieve and use that status data at a later time.

The network 2700 can enable user to monitor and/or control operation ofthe network devices 2702-2708. For example, the user can monitor andcontrol network devices by interacting with a visual interface of thegateway 110 (i.e., a web page for gateway 110) and/or a visual interfacerendered on display 2622 of the access device 108. In some embodiments,an application may be run on the access device 108. The application maycause the access device 108 to present a graphical interface (e.g., thedisplay 2622) that includes a visual interface for each network deviceaccessible on the network 2700.

The visual interface corresponding to a network device can provide a“status” of the network device. In certain embodiments, the status orstate of a network device can be indicated within the tile with textand/or graphically. The status of the network device can change based ontime (e.g., a period, an interval, or other time schedule). The statusof a network device may be any piece of information pertinent to thatparticular network device. The status of a network device may be anychangeable variable of that particular network device. For example, thestatus of a network device may include a state of the network deviceitself (e.g., on or off) or how the network device is situated withinthe network with respect to the other network and other network devicesthroughout the network. For example, the status of a network device mayrefer to the network device's proximity to another network device and/orits ability to communicate with another network device because of therelative signal strength between the two network devices. In certainembodiments, the status can include a value or some other informationindicating a unit of measure for a setting or an attribute related tooperation of a network device. The setting or the attribute can beadjustable within a range of values. For example, the network device canbe a light bulb and the status can include a value corresponding tobrightness (e.g., a percentage of total brightness) emitted by the lightbulb when the light bulb is powered-on. In another example, the networkdevice can be a motion sensor and the status can include a valuecorresponding to sensitivity of the sensor in a range of values between0 to 100 when the sensor is powered-on. In yet another example, thenetwork device can be a fan and the status can include a valuecorresponding to a speed of the fan on a scale of 0 to 100 when the fanis powered-on.

An application operating on the access device 108 can access the cache2602 to obtain information to display the visual interface for eachnetwork device 2702-2708 registered within the network 2700. FIG. 27Balso illustrates that each of the network devices 2702-2708 may includea storage device, such as a cache, for storing data, including dataregarding its own status and data regarding statuses received from theother devices within the network 2700.

As explained with reference to FIG. 2, the cache 2602 can contain aknown interface list 220 including interface information for different,known types of devices. As explained earlier, the known interface list220 can include a record for each network device known by the accessdevice 108 to exist on the network 2700. When an application is run onthe access device 108, the access device 108 can access the knowninterfaces 240 in the cache 2602 to present the display 2622. Thedisplay 2622 can present one or more visual interfaces, eachcorresponding to a network device known to exist on the network 2700.Each visual interface can be generated based on a visual interfacemodule corresponding to each network device on the network 2700. In anexample, the display 2622 can include a visual interface (e.g., a moduletile) for each network device in the network 2700 having an interface inthe known interface list 220.

The cache 2602 can also contain a known network device list 2720 with aDevice A (e.g., network device 2702) last known address 2724, a Device B(e.g., network device 2704) last known address 2726, and a Device C(e.g., network device 2706) last known address 2728. The known devicelist 2720 can list each device known by the access device 108 to existon the local area network 2700. Devices 2702-2706 can be present on theknown device list 220 because the access device 108 or another devicecapable of updating the cache 2602 of the access device 108 haspreviously discovered network devices 2702-2706.

In addition to address information, the cache 2602 can contain knownstatus information 204 about each network device in the known devicelist 2720. When the application is run on the access device 108, theaccess device 108 can access the known status information in the cache2602 to present a status display 2622. The access device 108 canpopulate each tile with an indicator representing the respective knownstatus information for each network device in the known device list2720. For example, the status display 2622 can include an indicatorrepresenting an “on” state for Device A 2702 and Device C 2706, and anindicator representing “off” for Device B 2704. The status display 2622can include an indicator of one or more attributes, one or moresettings, or a combination thereof related to operation of each networkdevice in the network 2700. For example, the status display can includea brightness of a bulb (e.g., a brightness of 56 in a range of valuesbetween 0 and 100) of the network device 2702 (e.g., a light bulb), avalue of sensitivity of a sensor (e.g., a value of 34 in a range ofvalues 0-100) for the network device 2704 (e.g., a motion sensor), and avalue of brightness (e.g., 65 percent brightness) for the network device2706 (e.g., a light bulb). Although shown as having a single indicatorfor an attribute or a setting related to operation of a network device,the status display 2622 can present a plurality of indicatorscorresponding to different attributes and/or settings related tooperation of a network device.

In some embodiments, the cache 2602 can include other information abouta network device. The other information can indicate a network device'sfirmware version, last known firmware update status, connectivity tocloud status, registration status (e.g., whether the network device hasa key or not), and other such information. The cache 2602 can includeinformation that could be used for troubleshooting. In embodimentsdescribed below, the access device 108 can access status informationfrom another other device on the network 2700 and can use thatinformation to update its own cache 2602, update the status display2622, and/or pass the information to the cloud network 114 and/or thegateway 110 for trouble shooting and/or storage.

In some embodiments, the cache 2602 may store information 2740 includingcriteria for determining actions to initiate at the network device. Thecriteria may be configurable by input received from a user via aninteractive area or interactive element of the display 2622. In someembodiments, the criteria may be received from another network device,such as one of the network devices 2702, 2704, 2706. The actionsinitiated at the network device based on the criteria may be in additionthose actions, which are related to operation of the network deviceitself. The actions may be initiated from the access device 108, any ofthe network devices 2702, 2704, 2706, other devices within the network2700, and/or devices outside the network 2700. The criteria may indicatewhen actions are to be performed. The criteria may indicate events basedon which actions may be initiated. Such events may be related tointeraction with an interface (e.g., a graphical interface or a physicalinterface) corresponding to a network device. An interface may controloperation of a network device. In some embodiments, interaction with theinterface may initiate additional actions in addition to controllingoperation of a network device corresponding to the interface.

In some embodiments, the criteria may be determined by a computingsystem implemented by a third party. The computing system may implementa service that enables a user to configure the criteria (e.g., one ormore rules) which can be used to determine actions to control operationof a network device, e.g., the network devices 2702-2708. In someembodiments, the computing system can determine actions to be performedbased on satisfaction of the criteria. The third party service mayenable a user to configure actions to be performed upon satisfaction ofa rule.

The computing system of the third party may be “outside” the local areanetwork 2700 such that it is remotely located (in an operational sense,a spatial sense, and/or a geographical sense) from network devices2702-2708, the gateways 110 or 112, and/or the cloud network 114. Thecomputing system may be configured to exchange communications with thecloud network 114.

In some embodiments, the computing system of the third party maydetermine whether a rule is satisfied based on information related tooperation of a network device in the network 2700. Cloud network 114 maycommunicate information related to operation of a network device (e.g.,a change in a state of the network device) to the computing system ofthe third party. The computing system may analyze one or more rulesconfigured by a user to determine whether a rule has been satisfiedbased on information received from cloud network 114. Upon satisfactionof a rule, the computing system may communicate information (e.g., aninstruction) to cloud network 114 indicating the rule that has beensatisfied and one or more actions to be performed. The information maycause operation of one or more network devices to be adjusted (e.g.,change an attribute or a setting related to operation of a networkdevice).

Actions may include controlling operation of one or more network devices(e.g., the network device 2702, the network device 2704, or the networkdevice 2706). Actions may include adjusting settings and/or attributesrelated to controlling operation of network devices. In someembodiments, actions may be specific to network devices, such as a typeof network device, a group of network devices, combinations thereof, orother criteria associated with network devices. In some embodiments, anadjustment to a setting or an attribute of a network device may bespecific based on a type of network device.

In some embodiments, the access device 108 may send communications toother devices, either on the network 400 or outside the network 400. Thecommunications may be sent in response to initiation of an action basedon the information 2740. The communications may include notifications,operations, actions, instructions, and/or data. The content of thecommunications may be determined based on the information 2740. Forexample, the access device 108 may send a notification to networkdevices in response to satisfaction of criteria (e.g., a rule) indicatedby the information 2740 associated with the network devices. Thenotification may be sent to a destination indicated by the information2740. In another example, the access device 108 may send an instructionindicating an action to be initiated based on satisfaction of criteriaindicated by the information 2740. Examples of actions are describedfurther with reference to FIGS. 28 and 29.

In some embodiments, the access device 108 may determine actions toperform based on input received from interaction with an interactivearea of the display 2622 on the access device 108. The access device 108may process the input to determine whether criteria indicated by theinformation 2740 are satisfied. In some embodiments, the input maycorrespond to a network device, such that the access device 108determines whether criteria corresponding to the network device havebeen satisfied. The access device 108 may perform actions in response todetermining that criteria are satisfied. As explained above, thecriteria may correspond to rules, modes of operation, and/ornotifications. The actions may include those, for example, describedwith reference to FIG. 27A.

In some embodiments, the actions may include sending communications to anetwork device (e.g., the network device 2702, the network device 2704,or the network device 2706). The communications may include data thatindicates which actions are to be performed by the network device. Anetwork device (e.g., the network device 2702, the network device 2704,or the network device 2706), which receives the data, may process thedata to determine which actions it is to perform. In some embodiments,the network devices 2702, 2704, 2706, may store in their respectivecache other criteria (e.g., rules). The actions performed based on thedata received by the network device, may trigger other actions to beperformed based on criteria stored locally in a network device's cache.Similarly, the access device 108 may receive data from any device,either on the network 400 or outside the network 400. The data mayindicate actions for the access device 108 to perform. The access device108 may process the data to determine the actions. The access device 108may use the data to determine whether criteria in the information 2704are being satisfied.

In some embodiments, the information 2740 may be stored in associationwith one or more network devices. The cloud network 114 may communicatethe information 2740 to devices, e.g., the access device 108. Theinformation 2740 may be received from a device remotely located from thenetwork 2700 (e.g., not on the network 2700) or within the network, suchas any one of the network devices 2702, 2704, 2706. In some embodiments,the information 2740 may be received from any of the network devices2702, 2704, 2706 on the network. The information 2740 may becommunicated to other network devices 2702, 2704, 2706 on the network2700 and/or the cloud network 114. In some embodiments, the information2740 may be generated based on input received from a user via a device,e.g., the access device 108. The information 2740 may include criteriaindicating how other devices are to operate. The criteria may indicateactions to be performed by other devices. Examples of techniques forreceiving input specifying the criteria in the information 2740 aredescribed further with reference to FIGS. 33-35.

The information 2740 may include, among other types of criteria,rule(s), mode(s), scene(s), and/or notification(s), in association witha device. For example, in FIG. 27A, Device A 2702 may be associated withstored information 2744. Similarly, Device B 2704 and Device C 2706 maybe associated with stored information 2746 and stored information 2748,respectively. The stored information 2744, 2746, 2748 corresponding tothe network devices 2702, 2704, 2706 may include rule(s), mode(s),and/or notification(s). Actions may be determined based on suchinformation stored in association with a network device. As such,actions that may be performed may be network device specific based onthe stored information. Examples of information (e.g., rules, scenes,modes, and/or notifications) that may be stored in the cache 2602 aredescribed further with reference to FIGS. 6 and 7.

In some embodiments, the information 2740 may include criteria such asrules. A rule may indicate criteria for performing actions. The criteriamay be based on a schedule, a time of day, events, input, a state ofnetwork devices, combinations thereof, or any other information that canbe used to determine what actions to perform and when to perform thoseactions. For example, a rule may indicate actions for controllingoperation of network devices. The criteria in a rule may indicate theactions to perform, when to perform those actions, and/or how to performthose actions.

In some embodiments, the rules may map different types of inputcorresponding to actions that may be performed for a network device. Thetypes of input may be defined based on input received at an interface,such as a tap, a click, a motion, a swipe, a movement, etc. The types ofinput may be based on a distance of movement corresponding to the inputand/or an amount of time the input is received. In some embodiments,actions may be mapped to data corresponding to input such ascoordinates, distance, motion, selections, taps, clicks, etc. Theactions corresponding to the types of input may include adjusting apower state of a network device, attributes related to operation of anetwork device, settings related to operation of a network device, orcombinations thereof.

In some embodiments, the information 2740 may indicate modes. A mode maycorrespond to a desired configuration of attributes related to operatingnetwork devices and performing actions (e.g., sending notifications andmanaging control of network devices). For example, in a homeenvironment, modes may include home modes, away modes, sleep modes, orother modes related to configurations for using network devices in ahome environment. Modes may be defined by criteria such as a schedule(e.g., a time and/or a date), user preferences related to use of certaintypes of network devices (e.g., lights, appliances, office equipment,etc.), settings and/or attributes related to operation of networkdevices, or other criteria for performing actions. Examples of modes aredescribed with reference to FIG. 28. Modes may be chosen using agraphical interface, examples of which are described with reference toFIGS. 33-35.

In some embodiments, the information 2740 may include criteria aboutsending notifications. The criteria may indicate when to sendnotifications, where to send notifications, and/or how to sendnotifications. A notification may include information that notifies auser about occurrences of events. The events may relate to operation ofthe network devices (e.g., the network devices 2702, the network device2704, or the network device 2706) or users operating network devices.The network devices may be on or outside the network 2700. Anotification may be sent by the access device 108 upon satisfaction ofcriteria for the notification. The criteria may be indicated by rules,modes, and/or an occurrence of events related to network devices.Criteria about a notification may indicate a destination for sending anotification. The destination may include network devices, users, adevice address, addresses (e.g., mailing address or email address),contact information, telephone number, email addresses, user names, anyinformation that can identify a recipient, or a combination thereof. Anotification may be generated in response to occurrence of one or moreevents that satisfy the criteria. A notification may be sent to adestination as a message and/or data, including details about one ormore events. In some embodiments, a notification may includeinstructions, operations, and/or requests to perform other actions thatmay provide a notification to user at a device or cause a notificationto be made. The notification may be displayed at a device. For example,a notification may be a message displayed at a device. The notificationmay include a visual indication displayed on a device, audio output by adevice, other type of notifications that can be sensed or perceived by auser, or combinations thereof.

FIG. 27B illustrates an example of a network 2700, according toembodiments of the present invention. The local area network 2700 mayinclude network device 2702, network device 2704, network device 2706,network device 2708, and access device 108. FIG. 27 also illustratesthat one or more network devices 2702-2708 and/or access device 108 mayinclude a storage device, such as a cache, for storing data, includingdata regarding its own status and data regarding statuses received fromthe other devices within local area network 2700. For example, accessdevice 108 may, after being powered up, broadcast/send its status tonetwork device 2708 via communication 2734. Network device 2708 maystore the status data received from access device 108 until the nexttime access device 108 updates its status by sending new/updated statusdata to network device 2708. Cache may be used for storage withinnetwork devices 2702-2708 and/or access devices within the local areanetwork 2700 so that each of the devices may be able to quickly retrievethe data it needs from storage. An application operating on the accessdevice 108 can access the cache to obtain information to display thevisual interface for each network device 2702-2708 registered within thenetwork 2700. Although a caching device may be used to store such datawithin the network and/or access devices within the local area network2700, other types of storage may be used.

The cache can contain a known interface list including interfaceinformation for different, known types of devices. The known list caninclude a record for each network device known by the access device 108to exist on the network 2700. When an application is run on the accessdevice 108, the access device 108 can access the known interfaces in thecache to present the display of access device 108. The display canpresent one or more visual interfaces, each corresponding to a networkdevice known to exist on the network 2700. Each visual interface can begenerated based on a visual interface module corresponding to eachdevice on the network 2700. In an example, the display can include avisual interface (e.g., a module tile) for each device in the network2700 having an interface in the known interface list.

The cache can also contain known status information about each networkdevice in the known device list. When the application is run on theaccess device 108, the access device 108 can access the known statusinformation in the cache to present a status display. The access device108 can populate each tile with an indicator representing the respectiveknown status information for each device in the known device list. Thestatus display can include an indicator of one or more attributes, oneor more settings, or a combination thereof related to operation of eachdevice in the network 2700. For example, the status display can includea speed of a fan (e.g., a fan speed of 56 in a range of values between 0and 100) of the network device 2702 (e.g., a fan), a value ofsensitivity of a sensor (e.g., a value of 34 in a range of values 0-100)for the network device 2704 (e.g., a motion sensor), a value ofbrightness (e.g., 65 percent brightness) for the network device 2706(e.g., a light bulb), and a value of temperature (e.g. a slow cooker).Although shown as having a single indicator for an attribute or asetting related to operation of a network device, the status display canpresent a plurality of indicators corresponding to different attributesand/or settings related to operation of a network device.

In some embodiments, the cache can include other information about anetwork device. The other information can indicate a device's firmwareversion, last known firmware update status, connectivity to cloudstatus, registration status (e.g., whether the network device has a keyor not), and other such information. The cache can include informationthat could be used for troubleshooting. In embodiments described below,the access device 108 can access status information from another otherdevice on the network 2700 and can use that information to update itsown cache, update the status display, and/or pass the information to thecloud network 114 and/or the gateway 110 for trouble shooting and/orstorage.

Even though each network device may know and store (e.g. in cache) thestate of each other network device within local area network 2700, anetwork device may not know when another network device changes status(e.g. turns/powers off). However, network devices and/or access deviceswithin local area network 2700 may broadcast/send any updates in itsstatus to other devices on the network. For example, if network device2702 changes status, it may send status data to the other networkdevices, such as network devices 2704, 2706 and 2708 and to accessdevice 108. However, network device 2702 may not know which devices toupdate since the other devices may change statuses periodically (e.g.turn off).

Therefore, a network or access device may subscribe to another networkor access device within local area network 2700. For example, networkdevices 2704, 2706 and 2708 and access device 108 may subscribe tostatus data notifications/updates from network device 2702. Such asubscription may be registered for upon initial connection with networkdevice 2702 when network device 2702 first enters local area network2700 or at any other time after network device 2702 has been associatedwith local area network 2700. Subscriptions may be controlled to lastindefinitely or may expire after a certain predetermined period of timeafter initial subscription. However, network devices may re-subscribe toanother network device before or after their previous subscription hasexpired.

Subscriptions between network device and/or access devices may beregistered, similar to registering a network device upon initialentrance into the local area network, including security registrationsdescribed herein with respect to FIGS. 1 and 2. For example, a networkdevice may send its unique security key, which it may have stored alongwith its network ID after being registered on the network, to a networkdevice to which it wants to subscribe. However, subscriptions may takeon many other forms, including sending a different form ofidentification to a network device to which a network device wants tosubscribe. However, subscriptions may take on many other forms,including sending a different form of identification to a network deviceto which a network device wants to subscribe.

Upon receiving a subscription from another network device or accessdevice, the device being subscribed to may store a list of the devicesthat subscribed to it. For example, network device 2702 may store a listof network devices 2704, 2706 and 2708 and access device 108 after thosedevices subscribe to network device 2702. Then, when network device 2702undergoes a change in status, network device 2702 may send that changein status to only the devices that had previously subscribed to it butwhere the subscription had not yet expired. Furthermore, according tosome embodiments, the subscription list of a network device may beautomatically updated if that device receives notification that anotherdevice has left the range of the local area network, either from thatdevice itself or from a different device. Therefore, the various deviceswithin a given local area network, such as network 2700, each containcontinuously updated statuses of each other device on the network andobtain those statuses and updates through direct communication withoutnecessary use of the cloud.

In some embodiments, network 2700 may include a device 2722. The device2722 may be located physically on one or more network devices 2702,2704, 2706, 2708 or in communication with one or more network devices2702, 2704, 2706, 2708. For example, the device 2722 may be incommunication with the device 2702. Such communication with the device2702 may occur through direct communication or wireless communication.Communication between the device 2722 and the device 2702 may usevarious protocols, such as Universal Plug and Play (UPnP), Bluetooth®,Zigbee®, Ultra-Wideband (UWB), WiFi-Direct, WiFi, Bluetooth® Low Energy(BLE), sound frequencies, and/or the like. In some embodiments, thedevice 2722 may communicate directly with the network devices 2702,2704, 2706, 2708. In some embodiments, the device 2722 may communicatewith the network devices 2702-2708 via the gateways 110, 112 (e.g.,communication) and/or the cloud network 114.

The device 2722 may be implemented as or included in a computing devicewith processing capabilities. For example, the device 2722 may includeone or more memory storage devices and one or more processors. In someembodiments, the device 2722 may include local cache 2730. A memorystorage device can be accessible to the processor(s) and can includeinstructions stored thereon which, when executed by the processor(s),causes the processor(s) to implement one or more operations disclosedherein as being performed by the device 2722. The device 2722 mayinclude network connection capability to access to a network, such asthe network 2700. The device 2722 may communicate with the network 2700in a manner similar to that of the access device 108 or the devices2702, 2704, 2706. The device 2722 may be located in a stationaryposition (e.g., mounted to a physical structure or on a device) or itmay be a mobile. For example, when stationary, the device 2722 may bemounted to a wall (e.g., a wall mounted control unit).

In some embodiments, the device 2722 may store, either in its cache 2730or memory, information to enable actions to be initiated at the device2722. The information may be similar to the information 2740. Theinformation may include criteria that indicate when actions are to beperformed. The criteria may indicate events based on which actions maybe initiated. Such events may be related to interaction with aninterface 2710 (e.g., a graphical interface or a physical interface)corresponding to the device 2722. In some embodiments, interaction withthe interface 2710 may initiate actions in addition those that may beperformed to control operation of the device 2722 configured to becontrolled by the interface 2710. The actions may be performed based onthe information stored in the cache 2730. Actions may include thosedescribed in the present disclosure, which may be performed by a device.

The device 2722 may include an interface 2710. The interface 2710 mayinclude any human-to-machine interface. For example, the device 2722 mayinclude a stand-alone interface such as an interface found on acomputing device (e.g., a cellular telephone, a smartphone, a homecomputer, a laptop computer, a tablet, a personal digital assistant(PDA), a computing device, a wearable device such as a smart watch, awall panel, a keypad, or the like), an interface that is built into anappliance or other device e.g., a television, a refrigerator, a securitysystem, a game console, a browser, or the like), a speech or gestureinterface (e.g., a Kinect™ sensor, a Wiimote™, or the like), an internetof things (IoT) device interface (e.g., an Internet enabled appliancesuch as a wall switch, a control interface, or the like). In someembodiments, the interface may include a capacitive touch areaconfigured to detect motions, gestures, devices, or combinationsthereof. The interface 2710 may be configured to control operation ofone or more devices 2702, 2704, 2706, 2708 to which the device 2722 maycommunicate.

Through interaction with the interface 2710, actions may be performedbased on criteria indicated by the information stored in cache 2730. Insome embodiments, actions may be performed based on information receivedfrom other devices, such as one of the devices 2702, 2704, 2706, 2708and/or the access device 108. The information may include criteriaindicating what actions are to be performed and when such actions are tobe performed. The interface 2710 may enable a user to specify criteria,e.g., the criteria in the information stored in cache 2730. Examples ofthe interface 2710 are described further with reference to FIG. 32. Theinterfaces described with reference to FIGS. 33-35 illustrate techniquesfor configuring criteria to perform actions. FIGS. 30 and 32 also showother examples of interfaces that may be implemented as the interface2710.

FIG. 28 illustrates examples of information 2800 including criteria ofinitiating actions according to some embodiments. Specifically, theinformation 2800 can be stored by a device including any one of theaccess device 108, the device 422, the network device 2702, the networkdevice 2704, the network device 2706, or the network device 2708). Forexample, the information 2800 can be included in the information 2740and/or the information 540. The information 2800 may be useful to enablea device to determine actions to perform, when to perform those actions,and/or how to perform those actions. FIG. 28 shows an examplerepresentative of the type of information including criteria that adevice may use to determine actions to be initiated based on inputreceived from interaction with an interface (e.g., a physical interfaceor a graphical interface). In some embodiments, the information 2800 mayinclude more or fewer actions and/or criteria. The information 2800 maybe included in or may correspond to rules associated with networkdevices.

The information 2800 shown in FIG. 28 is an example of criteria forinitiating actions at a device (e.g., a network device or an accessdevice). The actions may be initiated upon satisfaction of the criteria.The information 2800 may include multiple records, each recordcontaining action(s) 2810 and additional action(s) 2820 to initiatealong with the actions(s) 2810. An action 2810 may be initiated forinteraction with an interface (e.g., a graphical interface or a physicalinterface), such as the interface 410. The actions 2810 may be performedupon interaction with an interactive area of an interface. Theinteractive area may correspond to a one or more network devices. Theadditional action(s) 2820 may be performed based on extended orcontinued interaction with the interactive area. Extended or continuedinteraction may include interaction with the interactive area extendingbeyond a threshold time period, when additional action(s) are performed.The extended or continued interaction may occur for a longer period oftime and/or may include interaction different from network devices thana device which caused initiation of the actions 2810. The action(s) 2810may include adjusting or setting a state of operation (e.g., adjusting apower state for a network device). The additional action(s) 2820 mayinclude adjusting one or more attributes and/settings corresponding tooperation of one or more network devices. The attributes and/settingsmay include a power state related to operation of a network device. Theinformation 2800 may include criteria configured based on input receivedfrom a user. The input may indicate criteria for performing actions2820. In some embodiments, the action(s) 2810 may be performed for anetwork device corresponding to an interface and the additionalaction(s) 2820 may be initiated for the same network device and/or othernetwork devices or group of network devices.

In some embodiments, the actions (e.g., the actions 2810 or theadditional action(s) 2820) may be performed based on satisfaction ofcriteria including an activation time 2830 (e.g., a time period). Theactivation time 2830 may be pre-determined. The action(s) 2810 may beperformed based on receiving input corresponding to an interaction withan interface. The interaction may be received in an interactive areacorresponding to a network device. The actions 2810 may be performed forthe network device corresponding to the interactive area. Additionalaction(s) 2820 may be performed based on input corresponding tointeraction with the interactive area occurs for a time period thatsatisfies or exceeds threshold time period corresponding to theactivation time 2830. The activation time 2830 may correspond to thenetwork device, which corresponds to the interactive area. In someembodiments, multiple activation times may be configured for initiationof actions. A first activation time may correspond to a first thresholdwhen additional action(s) 2820 may be initiated. A second activationtime may correspond to a second threshold when other additionalaction(s) may be performed after the first activation time. Theadditional action(s) 2820 may include controlling operation of one ormore network devices according to attributes and/or settings indicatedby the information 2800. The additional action(s) 2820 may be chosenbased on the network device for which input was received from theinteractive area.

The information 2800 in FIG. 28 provides an example of criteriaindicating actions to be performed based on interaction with aninterface. The action(s) 2810 may be performed based on input receivedfrom interaction with the interface. In this example shown in FIG. 28,the actions 2810 include controlling a power state of a network device,such that an action is performed to adjust a network device to the powerstate indicated by the information 2800. The action(s) 2810 may bechosen based on a current state 2805 of an attribute (e.g., power state)of the network device to be adjusted. For example, for a light bulb, theaction 2810 may include adjusting a power state to “ON” when the currentpower state of the light bulb is “OFF,” or vice versa. Continuing withthe example, the additional action(s) 2820 may include controllingoperation of other network devices, such as network devices or group ofnetwork devices (e.g., an indoor lighting system, an exterior lightingsystem, or security sensor). The additional action(s) 2820 may beperformed to adjust network devices upon determining that interactionwith the interface occurs for at least the activation time 2830. Theactivation time 2830 may be chosen based on a current state of a networkdevice before the action(s) 2810 is performed. In one example, theadditional action(s) 2820 may include controlling a power state and/oradjusting one or more attributes (e.g., brightness) or settings of thenetwork devices. The attributes and/or settings shown as the additionalaction(s) may indicate the desired settings and/or attributes to beadjusted. In some embodiments, the additional action(s) 2820 may includecontrolling operation of different types of network devices, such as asensor and a light. In some embodiments, the additional action(s) 2820may be performed to control network devices regardless of their currentstate.

In some embodiments, a record (e.g., the record 2802 or the record 2804)in the information 2800 may indicate criteria for an access control2840. The access control 2840 may indicate one or more users that mayconfigure and/or initiate actions corresponding to an action. Forexample, the record 2804 may include two users, a first user (e.g.,“User 1”), such as a “Son,” and a second user (e.g., “User 2”), such asa “Daughter,” both of which may configure and initiate the actions inthe record 2804. In some embodiments, a device (e.g., the device 422)that processes the information 2800 may determine a user interactingwith the interface (e.g., the interface 410). In such embodiments, thedevice may determine whether actions in 2840 can be performed for thelight bulb based on whether the user has access as indicated by theaccess control 2840 of the record 2840.

In some embodiments, a record (e.g., the record 2802 or the record 2804)in the information 2800 may include information about notifications 2850that may be provided about the actions 2810 or the actions 2820 that maybe initiated for the record. The notifications 2850 may indicate theusers that can be notified regarding the actions performed for acorresponding record. In FIG. 28, the notifications 2850 may indicateone or more users that may be notified about activation of actions 2820.For example, the record 2804 may include a user, such as a user (e.g.,“User 3”) that may be provided with a notification when the actions 2820are initiated. The notifications may include sending a message the user.The message may be communicated to devices associated with a user, suchas an access device or a computing device. The message may include anemail, an SMS message, a video message, other types of communicationmessages, or combinations thereof. A notification may be useful toenable a user to become aware of the presence of others based on theactions performed in response to interaction with an interface. Thepresence of a user at a location may be inferred from a notificationabout an action performed with respect to actions 2810, 2820 that may beperformed at the location. In this example, the notifications may beuseful for a parent to learn when their son and/or daughter are presentat a location where actions 2810, 2820 are performed on the devices.

In some embodiments, the information 2800 may indicate modes ofoperation. For example, the information 2800 may indicate one or moremodes of operation corresponding to the actions 2810, 2820 that may beperformed. In one example, each record (e.g., the record 2802 and therecord 2804) may correspond to one or more modes. The modes may be usedto determine actions to be performed based on the information 2800.Criteria that may be received from a user may indicate modes ofoperation to be associated with actions. In some embodiments, a user mayindicate a mode of operation at an interface of a device, such as avisual interface presented on the display 2622 of the access device 108.The mode of operation, if selected before interaction with an interfacefor controlling a network device, may be used to determine the actions2810 and the actions 2820 that can be performed. In one example, therecord 2802 include a mode indicating when the actions 2810 and theactions 2820 in the record 2802 may be performed. In some embodiments,the information 2800 may include criteria such as a schedule (e.g., adate and/or a time) when actions 2810, 2820 corresponding to a recordmay be performed. Thus, the actions 2810, 2820 may be performed inresponse to interaction with an interface during a time corresponding tothe schedule.

FIG. 29 illustrates examples of information 2900 including criteria ofinitiating actions according to some embodiments. Specifically, theinformation 2900 may be stored by a device including any one of theaccess device 108, the device 422, the network device 2702, the networkdevice 2704, the network device 2706, or the network device 2708). Forexample, the information 2900 can be included in the information 2740and/or the information 540. The information 2900 may be useful to enablea device to determine actions to perform, when to perform those actions,and/or how to perform those actions. FIG. 29 shows an examplerepresentative of the type of information including criteria that adevice may use to determine actions to be initiated based on inputreceived from interaction with an interface (e.g., a physical interfaceor a graphical interface). In some embodiments, the information 2900 mayinclude more or fewer actions and/or criteria. The information 2900 maybe included in or may correspond to rules associated with networkdevices.

The information 2900 in FIG. 29 shows an example of criteria forperforming actions with respect to a variety of scenes 2910. In thisexample, actions 2920 and additional actions 2930 may be associated witha scene. In some embodiments, actions may be performed based on aselected scene. The information 2900 indicates states and/orsettings/attributes related to operation of network devices. As shown,actions may be performed to adjust network devices according to thestates and/or settings/attributes corresponding to a scene. The actionsmay be initiated based upon the criteria for a scene. In the exampleshown in FIG. 29, the information 2900 may include scenes such as amovie scene 2902, a normal scene 2904, a reading scene 2906, and adefault scene 2908. Any number of states, settings, and/or attributesmay be specified for network devices controlled for a scene. In someembodiments, network devices may be adjusted to an “OFF” power state,while others are adjusted for their settings/attributes in an “ON” powerstate.

Scenes may correspond to preferences of a user for performing actionsand controlling operation of network devices. The preferences may bebased on environmental preferences that are based on use of networkdevices. In the example provided, for example, a movie scene maycorrespond to preferences for operating network devices while a movie isbeing watched. For the movie scene 2910, the additional actions 2930 mayinclude adjusting network devices, such as a speakers, living roomlighting, and appliances (e.g., a popcorn maker). The additional actions2930 may include adjusting attributes and settings of these networkdevices. When a user interacts with an interface configured to control alight bulb for a time period satisfying the activation time, the actions2930 may be performed. In some embodiments, the scenes 2910 may includea default scene 2908. For example, the default scene may be activatedwhen interaction with an interface to control the light bulb lasts forat least the activation time.

The information 2900 may include multiple records 2902-2908, each recordcontaining action(s) 2920 and additional action(s) 2930 to initiateafter the action(s) 2920 are initiated. The actions 2920, 2930corresponding to a record may be performed based on a scene 2910 in thescene 2910 indicated in a record. Network devices, such as thoseindicated in the records, may be configured to operate according to thechosen scene. In some embodiments, a scene may be chosen via interactionwith an interface (e.g., an interface presented on the display 2622). Insome embodiments, a scene 2910 may be triggered upon satisfaction of anactivation time 2940 corresponding to a record. The activation time 2940may indicate a time when additional actions 2930 are initiated. Theadditional actions 2930 may be initiated when interaction with aninterface occurs for at least the activation time 2940. The interactionmay correspond to controlling a network device (e.g., the light bulb2708) identified in the actions 2920. The additional actions 2930 may beconfigured for the scene 2910. By interacting with an interface for theactivation time, the additional actions 2920 may be initiated accordingto the scene.

In some embodiments, notifications 2940 may be sent upon activation of ascene. For each of the scenes 2902-2908, notifications may be sent todifferent users. The user may be associated with information todetermine a destination (e.g., email address, device address, phonenumber, etc.) for delivery of a notification. The notification may besent either when a scene is activated, deactivated, or both.

Throughout FIGS. 30-35, various graphical interfaces (e.g., displays)and/or physical interfaces (e.g., a switch, a button, or the like) areshown for initiating and/or configuring actions in accordance with someembodiments. A physical interface may be on a device (e.g., the networkdevice 2702, the network device 2704, the network device 2706, or thenetwork device 2708) that is controlled using the physical interface. Aphysical interface may be on a device (e.g., the device 422) incommunication with one or more devices (e.g., the network device 2702,the network device 2704, the network device 2706, or the network device2708) that are controlled by the physical interface. A graphicalinterface may be displayed by an access device (e.g., the access device108) rendering one or more visual interfaces to enable control ofoperation of network devices (e.g., the network devices 2702, 2704,2706, 2708).

A device controlled by an interface (e.g., a physical interface or agraphical interface) may include or be communicatively coupled to anetwork device. Visual interfaces may be rendered upon execution of avisual interface module by an application on an access device (e.g., theaccess device 108). The visual interfaces may be shown with one or acombination of modular tiles, interactive areas, interactive elements,icons, links, command regions, windows, toolbars, submenus, drawers, andbuttons that are used to initiate actions, invoke routines, monitornetwork devices, configure network devices, control network devices, orinvoke other functionality. The initiated actions may include, but maynot be limited to, displaying a status of a network device, displaying acurrent state of a network device, displaying historical data associatedwith a network device, displaying a last known status of a networkdevice, adjusting one or more settings and/or one or more attributesrelated to operation of a network device, adjusting one or moresecondary settings of a network device, sending notifications,implementing rules that cause other actions to be performed, initiatingactions, or a combination thereof. For brevity, the differencesoccurring within the figures, as compared to previous or subsequentfigures, are described and illustrated below for purposes ofillustration.

FIG. 30 shows an example of interfaces presented by a device (e.g., theaccess device 108 or the device 422) according to some embodiments. Inparticular, FIG. 30 shows a display 3005 that presents a graphicalinterface with modular tiles 3002, 3004, 3006 for interacting withnetwork devices in a network. The modular tile 3002, modular tile 3004,and the modular tile 3006 may correspond to network device 2702, networkdevice 2704, and network device 2706, respectively. A modular tile mayenable interactions to monitor and/or control operation of a networkdevice corresponding to the modular tile. Interaction with a modulartile may enable a user to initiate one or more actions in addition tooperating a network device corresponding to the modular tile. Theadditional action(s) may offer enhanced functionality that may not beavailable from a typical interface (e.g., a graphical interface orphysical interface) that enables a user to control operation of anetwork device. The additional action(s) may include actions, forexample, the additional action(s) 2820 or the additional action(s) 2930.The FIGS. 33-35 described further below may enable a user to configuresuch additional action(s).

A modular tile may include interactive areas or interactive elements toenable interaction with the modular tile. The modular tiles 3002, 3004,3006 may each include respective icons 3080, 3082, 3084, and respectivenames 3040, 3042, 3044 corresponding to a network device. For example,the modular tile 3002 corresponds to the network device 2702 (e.g., alight bulb) and includes an interactive element depicted as a powerbutton 3030 for turning the network device 2702 on and off. In anotherexample, the modular tile 3004 corresponds to the network device 2704(e.g., a motion sensor) and includes an interactive element depicted asa power button 3032 for turning the network device 2704 on and off. Inanother example, the modular tile 3006 corresponds to the network device2706 (e.g., a light bulb) and includes an interactive element depictedas a power button 3034 for turning the network device 2706 on and off.An interactive element shown as a control (e.g., power button) may bedisplayed based on a state of the network device corresponding to thecontrol. An appearance of the interactive element may be altered to bevisibly illuminated, colored, shaded, bolded, animated, modified to adifferent shape, or a combination thereof. Although depicted as acorresponding to a single network device, each modular tile maycorrespond to multiple network devices, or a group of network devices.For purposes of illustration, a modular tile may be described herein asenabling control for operation of a network device. However, in someembodiments, a modular tile may enable control for operation of multiplenetwork devices, some or all of which may be different.

In some embodiments, a modular tile may be customizable by developersand/or manufacturers of the corresponding network devices. For, example,name 3044 of the modular tile 3004 is customized to indicate the name ofa specific type of sensor (e.g., “Security Sensor”). In additional oralternative embodiments, a modular tile may convey status informationabout a corresponding network device or network devices, including, butnot limited to, a firmware version, a last known firmware update status,status for connectivity to a cloud network, registration status (i.e.,an indication that the network device has a key or does not), a primarymode of the network device (i.e., on, off), a secondary mode of thenetwork device (i.e., standby, high, low, eco), other modes ofoperation, a schedule, a scene that is enabled using the network device,settings (e.g., speed or sensitivity) for the network device, one ormore attributes (e.g., brightness) related to operation of the networkdevice, additional actions that have been initiated based on interactionwith a modular tile, rules that may be in effect for the network deviceor related to operation of the network device, or a combination thereof.

Each of the modular tiles 3002, 3004, 3006 may be chosen and displayedbased on the type and/or capabilities of a network device (e.g., thenetwork devices 3002, 3004, 3006). The display 3005 may be flexible andmay dynamically display the most appropriate modular tile 3002 based onan environment in which the application executes. For instance, thedisplay 3005 may enables a mechanism in which a user may pre-configure amodular tile to display an appropriate message for a network device.

FIG. 30 shows that display 3005 may include interactive elements (e.g.,selectable icons and links) 3014, 3016, 3018, 3020, 3022, and 3024outside of the modular tiles 3002, 3004, 3006. For example, refresh icon3014 may be selected to refresh information presented in display 3005,such as, for example, status and state information displayed in themodular tiles 3002, 3004, 3006. For instance, one or more of thestatuses for each of the modular tiles 3002, 3004, 3006, respectively,may be refreshed on an automatic, periodic basis, in addition to beingmanually updated when refresh icon 3014 is interacted with (e.g.,selected).

In some embodiments, the display 3005 may include an edit link 3016.Interaction with the edit link 3016 may enable one or more of themodular tiles 3002, 3004, 3006 to be editable. For example, edit link3016 may be selected to sort or re-order the sequence of the modulartiles 3002, 3004, 3006 displayed in the display 3005. Upon interactionwith the edit link 3016, one or more of the modular tiles 3002, 3004,3006 may be interacted with to be disabled or removed. Devices icon 3018may be selected to cause a graphical interface to be presented with oneor more network devices that are discovered in a network.

In some embodiments, the display 3005 may include a rules icon 3020.Interaction with a rules icon 3020 may enable a graphical interface thatenables customization of display rules pertaining to the networkdevices. The graphical interface may provide one or more interactiveelements to control one or more rules related to control of the networkdevices (e.g., the network devices 2702, 2704, 2706). In someembodiments, interaction with the rules icon 3020 may enable a graphicalinterface to enable a user to configure rules, modes, notifications,additional actions, or combinations thereof, any or all of which may bestored locally on the device (i.e., the information 2740, theinformation 540, the information 2800, or the information 2900 stored onthe cache 2602) presenting the display 3005 or other network devices(e.g., the network devices 2702, 2704, 2706). The graphical interfacemay be presented in the display 3005 or may be shown separately from thedisplay 3005.

In some embodiments, the display 3005 may include a news icon 3022.Interaction with the news icon 3022 may enable a graphical interface tobe display which presents news including information associated with thenetwork devices and/or the applications. For instance, interaction withthe news icon 3022 may cause a graphical interface to be displayed inthe access device to present information related to the network devicescontrolled via the modular tiles 3002, 3004, 3006 and/or informationrelevant to the application, such as notifications of available tileupdates.

In some embodiments, the display 3005 may include a more icon 3024.Interaction with the more icon 3024 may provide one or more additionalgraphical interfaces to enable access to additional features forcontrolling and/or monitoring the network devices in a network. Forexample, an additional graphical interface may be displayed to controlone or more settings and/or attributes of any network devices connectedto a network. In another example, an additional graphical interface maybe displayed to configure secondary options to be initiated by aninterface used to control a network device. An example of an additionalgraphical interface is described with reference to FIG. 35.

The display 3005 may receive input to adjust one or more settings and/orone or more attributes related to operation of a network devicecorresponding to a tile presented in the display 3005. Thefunctionalities may include secondary functionalities, tertiaryfunctionalities, etc. beyond primary functionalities (e.g., powercontrol). The display 3005 may be interactive or may include interactiveareas configured to receive input to assist the user in controllingfunctionality, settings, and/or attributes of a network deviceidentified in the display 3005. The display 3005 may be interactive ormay include interactive areas to enable initiation of actions andadditional actions. The additional actions may include controllingoperation of network devices identified in the display 3005 as well asor in addition to other devices not shown.

Each modular tile 3002-3006 may be interactive to receive input. Theinput received via a modular tile may be used to control operation of anetwork device corresponding to the modular tile. The input may includeone or more contacts by a user. A contact may include a tap, a touch, apinch, a drag, a click, a swipe, a circular motion, a selection, acontinuous movement, other motions, other user interactions, usermovements, or combinations thereof. In some embodiments, the input maybe received as a single input or a series of inputs to control thenetwork device corresponding to the modular tile. The contact may occurin one or more directions on the display 3005 or a modular tile.

In one illustrative example, the display 3005 can receive input via aninteractive area 3050 of the modular tile 3006. The input may be used tocontrol operation of the network device 306 corresponding to the modulartile 3006. The interactive area 3050 may coincide with an interactiveelement (e.g., the button 3034). The input may be received during a timeperiod (e.g., a first time period). The input may include datacorresponding to contact with the interactive area 3050. The contact maybe continuous during the time period. The first time period maycorrespond to a time period when interaction within the interactive area3050 began until an activation time. The input received during the timeperiod may cause an action (e.g., an action) to be performed. The actionmay include operation of the network device 2706. The input maycorrespond to an adjustment for controlling operation of the networkdevice 2706. The network device 2706 may be controlled based on theadjustment. For example, a power state of the network device 2706 may bechanged in response to the input. In this example, the network device2706 is turned to an “on” power state.

The adjustment for controlling operation of the network device 2706 mayinclude adjusting an attribute or an setting (e.g., brightness) of thenetwork device 2706. The adjustment to the attribute can be based on anamount of the input, a length of time when the input is received, adistance of movement corresponding to the input, a direction of theinput, or any other data obtained from the input. The network device canbe adjusted from a last known state of operation or last known state ofan attribute or setting.

In some embodiments, the display 3005 may be modified to a display 3010in response to the input received during the first time period. Thedisplay 3010 may be different from the display 3005 based on the inputreceived during the first time period. Continuing with the exampleabove, the display 3005 may be modified to cause the button to bedisplayed with an indication that the network device 2706 has a powerstate of “ON.”

Input may be received via the display 3010 during a time period (e.g., asecond time period) occurring after the first time period. The secondtime period may occur immediately after the first time period. In someembodiments, the second time period may begin before an activation time(e.g., the activation time 2830) has been reached. The activation timemay correspond to a threshold time or a threshold time period, whichwhen reach, additional actions may be performed based on the inputreceived via the display 3010. The second time period may end at a timethat occurs at the threshold time period has been reach or exceeded. Theinput may be received via the interactive area 3050 in which input wasreceived during the first time period. The input may correspond tocontinuous contact with the interactive area through the first timeperiod and the second time period. The input received via theinteractive area during the second time period may cause one or moreadditional actions (e.g., the additional action 2820 or the additionalactions 2930) to be performed. In some embodiments, the additionalactions may be performed when the threshold time period has beenreached. In the example shown in FIG. 30, the additional actions mayinclude changing a power state to “OFF” for the network device 2702 andthe network device 2704 corresponding to the modular tiles 3002 and3004, respectively. The icons 3030 and 3032 are shown as unshaded toreflect that the network device 2702 and the network device 2704 areoff. Therefore, the additional actions may be performed if the thresholdtime period has been reached at any point during the second time period.

In some embodiments, the display 3010 may be modified to a display 3015in response to the input received during the second time period. Thedisplay 3015 may be different from the display 3010 based on the inputreceived during the second time period. For example, the modular tile3002 and/or the modular tile 3004 may be modified if a status of acorresponding network device changes in response to the action. Asexplained above, an additional action may include controlling operationof network devices such as the network device 2702 or the network device2704 corresponding to the modular tile 3000 and the modular tile 3002,respectively.

FIG. 31 shows another example of interfaces presented by a device (e.g.,the access device 108 or the device 422) according to some embodiments.The display 3105 may include elements described with reference to FIG.30, such as elements included in the display 3005. The display 3105includes a graphical interface with the modular tiles 3002, 3004, 3006.The modular tile 3002, modular tile 3004, and the modular tile 3006 maycorrespond to network device 2702, the network device 2702, and thenetwork device 2706, respectively. The example shown in FIG. 31 alsoenables a user to initiate one or more additional actions besidescontrolling operation of a network device identified in the display. Forexample, interaction with a modular tile may enable a user to initiateone or more additional actions in addition to controlling operation of anetwork device corresponding to the modular tile.

In the example of FIG. 31, input may be received via a modular tile,such as the modular tile 3006, in the display 3105. The modular tiles3002-3006 may be interactive to receive input. For example, input may bereceived via interaction with an area (e.g., an area 3150) of themodular tile 3006. Similar to the example described with reference toFIG. 30, the input may be used to control operation of the networkdevice 2706 corresponding to the modular tile 3006. The input may bereceived during a time period (e.g., a first time period). The input mayinclude data corresponding to interaction with the area 3150. Theinteraction with the area 3150 may include one or more contacts with thearea 3150 by a user. A contact may include a tap, a touch, a click, aswipe, a motion, a selection, a continuous movement, other motions,other interactions, or combinations thereof.

In the example shown in FIG. 31, the input may correspond to a swipemotion of a user's finger from the interactive area 3150 to a differentinteractive area 3155 in the tile 3006 during the first time period. Theinput may include data about the movement in the tile to a differentinteractive area 3155. The input corresponding to the interaction may bereceived during the first time period. The first time period may endbefore, after, or at a beginning of a threshold time period. A thresholdtime period may be based on an activation time (e.g., the activationtime 2830). The activation time may be based on information (e.g., theinformation 2740, the information 2800 or the information 2900) storedat the device presenting where the interaction occurs. The informationmay indicate an activation time corresponding to input received foroperation of the network device corresponds to the tile 3006. The inputreceived during the first time period may cause an action (e.g., anaction) to be performed. The action may include operation of the networkdevice 2706. The input may correspond to an adjustment for controllingoperation of the network device 2706. The network device 2706 may becontrolled based on the adjustment. For example, a power state of thenetwork device 2706 may be changed in response to the input. In thisexample, the power state for the network device 2706 is adjusted to an“ON” state.

The adjustment for controlling operation of the network device 2706 mayinclude adjusting an attribute or an setting (e.g., brightness) of thenetwork device 2706. The adjustment to the attribute can be based on anamount of the input, a length of time when the input is received, adistance of movement corresponding to the input, a direction of theinput, or any other data obtained from the input. The network device2706 can be adjusted from a last known state of operation or last knownstate of an attribute or setting.

In some embodiments, the display 3105 may be modified to a display 3110in response to the input received during the first time period. Thedisplay 3110 may be different from the display 3105 based on the inputreceived during the first time period. Continuing with the exampleabove, the display 3105 may be modified to cause the button to bedisplayed with an indication that the network device 2706 has a powerstate of “ON.”

Input may be received via the display 3110 during a time period (e.g., asecond time period) occurring after the first time period. The secondtime period may occur immediately after the first time period. In someembodiments, the second time period may begin before an activation time(e.g., the activation time 2830) has been reached. The activation timemay correspond to a threshold time or a threshold time period, whichwhen reached, additional actions may be performed based on the inputreceived via the display 3110. The second time period may end at a timethat occurs at the threshold time period has been reach or exceeded. Theinput may be received via the interactive area 3155 in which input wasreceived during the first time period. The input may correspond tocontinuous contact with the interactive area 3155 during the second timeperiod. In one example, touching the interactive area 3155 immediatelyafter the first time may indicate continuous contact with theinteractive area during the second time period. The input received viathe interactive area during the second time period may cause one or moreadditional actions (e.g., the additional action 2820 or the additionalactions 2930) to be performed. In some embodiments, the additionalactions may be performed when the threshold time period has beenreached. Therefore, the additional actions may be performed if thethreshold time period has been reached at any point during the secondtime period.

In some embodiments, the display 3110 may be modified to a display 3115in response to the input received during the second time period. Thedisplay 3115 may be different from the display 3110 based on the inputreceived during the second time period. For example, the modular tile3002 and/or the modular tile 3004 may be modified if a status of acorresponding network device changes in response to the action. Similarto the example shown in FIG. 30, an additional action may includecontrolling operation of network devices such as the network device 2702or the network device 2704 corresponding to the modular tile 3000 andthe modular tile 3002, respectively.

FIG. 32 shows another example of interfaces presented by a device (e.g.,the device 422) according to some embodiments. Specifically, FIG. 32illustrates various different types of interaction with a physicalinterface 3202 (e.g., the interface 410) of a device (e.g., the device2722). In this example, the physical interface 3202 is a control switchwith one or more interactive areas (e.g., a touch pad) and/orinteractive elements (e.g., buttons). The interface 3202 may beconfigured to control operation of at least one network device (e.g.,the network device 2706).

FIG. 32 illustrates an example of interaction with the physicalinterface 3202 during different stages occurring two time periods, afirst time period and a second time period. Each display shown below thephysical interface shows a status of a network device controlled byinteraction with the physical interface 3202 during the differentstages.

Beginning at first stage, a user may interact with one of many differentinteractive areas to control operation of a device corresponding to thephysical interface 3202. Interaction with an interactive area mayinclude one or more contacts with the interactive area. In this example,the device corresponding to the physical interface 3202 is the networkdevice 2706, labeled as “Exterior Lighting” in the display 3205. Thephysical interface 3202 may have one or more interactive elements suchas a button 3204. The physical interface 3202 may have one or moreinteractive areas 3208, 3214 with which the user can physicallyinteract. The interactive areas 3208, 3214 may exist on the physicalinterface 3202. The physical interface 3202 shown is one of manyembodiments of a physical interface.

In some embodiments, physical interface 3202 may be configured to detecta presence of another device (e.g., access device 108 or other type ofdevice) within proximity of the physical interface 3202. The inputreceived based on detecting the presence of another device may be usedto control operation of a network device corresponding to physicalinterface 3202. In some embodiments, a presence of a device may cause asetting or an attribute of a network device to be adjusted. A setting oran attribute may be adjusted with respect to a distance of a devicedetected as being present near physical interface 3202. For example, thebrightness of a light bulb may be adjusted based on the proximity of adevice to physical interface 3202. In another example, a power state ofa network device may be adjusted based on detection of the presence of adevice. In one example, physical interface 3202 could sense a person inpossession of a device coming into the room causing a lighting system tobe activated upon entry of the person. The lighting system can beadjusted (e.g., deactivated) as the person leaves the room with thedevice.

In some embodiments, physical interface 3202 may include or be coupledto one or more sensors (e.g., a temperature sensor, a pressure sensor, alight sensor, a motion sensor, or combinations thereof). A sensor maydetect the presence of a person or an object (e.g., a device). Inputreceived via physical interface 3202 may be used to control operation ofa network device. For example, the presence of an object or theoccurrence of an event based on detected input via a sensor may causeoperation of a network device to be controlled. In some embodiments, anattribute or a setting of a network device may be adjusted accordinglybased on the type of input received from a sensor. A network device maybe adjusted based on an occurrence of an event corresponding to input, adistance of an object that is detected, a type of event that is detectedby a sensor, other conditions based on input received from a sensor, orcombinations thereof. For example, a light system may be activated ordeactivated by a gesture from a person detected by physical interface3202.

Specifically in FIG. 32, an example is shown of controlling operation ofa network device based on interaction by a user with physical interface3202. During the first stage, a display 3205 may be presented at anaccess device (e.g., the access device 108) showing a status of networkdevices including the device corresponding to the physical interface3202. The display 3205 may include elements described with reference toFIG. 30. The network device 2706 is shown as having a power state “OFF”indicated by the icon 3034. A user may interact with any one of theinteractive areas of the physical interface 3202 in the first stageduring the first time period. A second stage may be reached when a usercontinues interaction with the physical interface 3202 during the firsttime period. In one example, a user may press on the interactive area3208 during the first time period. In another example, a user mayinteract with the physical interface 3202 by moving a finger from oneinteractive area 3214 to another interactive area 3216. In someembodiments, a user may repeatedly press or hold down the interactiveelement 3204 in one direction to control the network device 2706.

A user may reach the second stage by interacting with the physicalinterface 3202 for the entirety of the first time period. A user maycomplete interaction by stopping contact with the physical interface3202. In some embodiments, a user may make continuous contact with thephysical interface for a second time period until a third stage isreached. The continuous contact may be uninterrupted from interactionduring the first time period. The second time period may occurimmediately after the first time period.

Upon reaching the second stage, the network device 2706 including thephysical interface 3202 may process the input received during the firsttime period and determine an action for controlling operation of thenetwork device 2706. The network device 2706 may perform the action. Thenetwork device 2706 may store an updated status. The updated status maybe communicated to an access device to update the display 3205. Thedisplay 3205 may be modified to the display 3210 to indicate that thenetwork device 306 has changed to a power state of “ON.” The icon 3034may be modified to reflect the updated status.

A user may continue interaction with the physical interface 3202 at thesecond stage through the second time period. As explained above, in someembodiments, the user may continue interaction at any one of theinteractive areas 3208, 3214, 3212. The interaction may continue fromthe second stage to the third stage during the second time period.

The network device 2706 may process the input to determine whether anadditional action has been configured corresponding to the interactionduring the second time period. In some embodiments, the network device2706 may determine whether the threshold time period has occurred orbeen exceeded. In some embodiments, the threshold time period may bereached during the second time period. Upon determining that a user hasinteracted with the physical interface for the threshold time period,the network device 2706 may perform additional actions that have beenconfigured. In this example, the additional actions include changing apower state of other devices, such as the network devices correspondingto the modular tiles, 3002, 3004. The additional actions may includechanging the power state to “OFF” for the network devices correspondingto the modular tiles 3002, 3004. The display 3210 may be updated topresent the display 3215 during the third stage. The modular tiles 3002,3004 are shown as updated with the icon 3030 and the icon 3032 modifiedto indicate an “OFF” power state.

FIG. 33 shows an example of a graphical interface presented by a deviceaccording to some embodiments of the present disclosure. Specifically,FIG. 33 shows a display 3305 presented by an access device. Interactionwith the display 3305 may enable a user to initiate and configureadditional actions in accordance with some embodiments. The display 3305may include elements described with reference to FIG. 30, such aselements included in the display 3005. The display 3305 includes agraphical interface with the modular tiles 3002, 3004, 3006. The modulartile 3002, modular tile 3004, and the modular tile 3006 may correspondto network device 2702, the network device 2704, and the network device2706, respectively. Interaction with a modular tile may enable a user toinitiate one or more actions in addition to controlling operation of anetwork device corresponding to the modular tile.

In some embodiments, a modular tile may include one or more interactiveelements to cause the display 3305 to be modified to present one or morevisual interfaces. For example, the modular tile 3006 may include aninteractive element 3304. Interaction with the interactive element 3304may cause a visual interface 3302 to be presented by the access deviceshown in FIG. 33. The display 3305 may be modified to a display 3315 topresent the visual interface 3302 as shown in FIG. 33. The display 3315may include some elements of the display 3305.

The visual interface 3302 may include one or more interactive elementsto enable configuration of actions (e.g., additional actions) for one ormore of the modular tiles 3002-3006 and additional actions. For example,interactive elements shown in the visual interface may enableconfiguration of actions to be initiated for interaction with themodular tile 3006. An interactive element 3310 in the visual interface3302 may enable configuration of actions to be performed in response tointeraction with the modular tile 3006. Interaction with the interactiveelement 3310 to an “ON” state may cause all or a combination of theinteractive elements 3312, 3314, 3316, 3318, 3320, 3322, 3324, 3326,3328 to be enabled for interaction in the display 3310. Additionalactions may be determined based on criteria determined from input to oneor more of the interactive elements 3314, 3316, 3318, 3320, 3322, 3324,3326. The additional actions and/or the criteria for performingadditional actions may be stored by a device, e.g., the access device108, the device 422, or any of the network devices 2702, 2704, 2706.

The visual interface may include the interactive element 3312 (e.g.,“Activation Time”). Interaction with the interactive element 3312 mayenables a user to indicate a time period (e.g., an activation time) forinitiating additional actions. In the example shown in FIG. 33, theactivation time is configured to 5 seconds. As explained earlier,interaction with an area in the modular tile 3006 occurring for longerthan the activation time may cause additional actions to be initiated.

The visual interface 3302 may include the interactive element 3314(e.g., “Notification(s)”). Interaction with the interactive element 3314may enable a user to input notifications to be sent. Although not shown,the interactive element 3314 or other interactive elements may bepresented to enable a user to indicate what type of notifications tosend, when to send the notifications, and/or the criteria for sendingnotifications. The input may include information about of users,devices, addresses, contact information, combinations thereof, or anyother information that can be used to determine a destination wherenotifications are to be sent. As described above, an additional actionmay include sending a notification to users. The notification may besent to a user via one or more communication techniques that enable thenotification to be received by the user. For example, a notification maybe sent as an SMS message to devices associated with a user. Thenotification may include information about or relating to additionalactions that are initiated via interaction with the modular tile 3006.The notification may include details about the additional actions thatare performed, the actions that are performed, a network devicecorresponding to the modular tile 3006 by which the secondarynotification was initiated, a time when the actions are performed, orany other information related to the additional actions that areinitiated.

The visual interface 3302 may include the interactive element 3316(e.g., “Access Control”). Interaction with the interactive element 3316may enable a user to indicate criteria related to access control foractions. The interactive element 3316 may enable receipt of informationindicating who (e.g., users or devices) that may initiate actions. Forexample, the interactive element 3316 may enable a user to indicate theusers that are permitted to initiate actions via the modular tile 3006.The interactive element 3316 may enable a user to indicate othercriteria for enforcing access control. The criteria may include usernames, user email addresses, contact information, or the like.

The visual interface 3302 may include the interactive element 3318(e.g., “Mode(s)”). The interactive element 3318 may enable a user toconfigure modes. The modes may include, among many others, away modes,home modes, and/or sleep modes. As described in this disclosure, a modemay indicate preferences or criteria for performing actions, such ascontrolling operation of network devices and sending notifications. Themodes may be configured in a separate graphical or visual interface.

The visual interface 3302 may include the interactive element 3320(e.g., “Device(s) Settings/Attributes”). The interactive element 3320may enable a user to indicate device settings and/or attributes to beadjusted. Actions may include adjusting devices according to the devicesettings and/or attributes configured using the interactive element3320. For example, the interactive element 3320 may be interactive toenable a user to indicate criteria, such as settings and attributes fora network device or group of network devices. The network device(s) maybe adjusted to the settings and/or attributes as an action. In someembodiments, the interactive element 3320 may include a control 3324 toenable selection of a network device or group of network devices toadjust. The interactive element 3320 may include a control 3326 toenable selection of an attribute or a setting to adjust for a networkdevice or group of network devices. The interactive element 3320 mayinclude a control 3328 to enable selection of criteria (e.g., a value ora state) to set for an attribute or a setting chosen using the control3326. The interactive element 3320 may enable a user to configure anaction for an adjustment of multiple network devices or multiple groupsof network devices based on settings and/or attributes.

The visual interface 3302 may include an interactive element 3322, whichenables one or more interactive elements, not shown in the visualinterface 3302, to be displayed in the visual interface 3302. Thedisplay 3315 may show more or fewer interactive elements to configure anaction. In some embodiments, multiple visual interfaces, such as thevisual interface 3302, may be shown in the display 3315. The visualinterface 3302 may enable multiple actions to be controlled, eachcorresponding to a different modular tile. The visual interface mayenable multiple actions to be configured for a single modular tile. Insome embodiments, each action may be configured for a differentactivation time. In the example shown, multiple actions may be performedaccording to the criteria indicated by the interactive elements 3312,3314, 3316, 3318, 3320, 3322, 3324, 3326, 3328.

The visual interface 3302 may include an interactive element 3306.Interaction with the interactive element 3306 may cause the display 3315to be modified such that the visual interface 3302 is no longer visibleor hidden in the display 3315.

FIG. 34 shows an example of a graphical interface presented by a deviceaccording to some embodiments of the present disclosure. Specifically,FIG. 34 shows an example of a display 3405 presented by an access deviceaccording to some embodiments. The display 3405 may be used to initiateand configure actions in accordance with some embodiments. The display3405 may include elements described with reference to FIG. 30, such aselements included in the display 3005. The display 3405 includes agraphical interface with the modular tiles 3002, 3004, 3006. The modulartile 3002, modular tile 3004, and the modular tile 3006 may correspondto network device 2702, the network device 2704, and the network device2706, respectively. Interaction with a modular tile may enable a user toinitiate one or more actions in addition to controlling operation of anetwork device corresponding to the modular tile.

In the example shown in FIG. 34, the display 3405 may include the visualinterface 3402. The visual interface 3402 may be displayed byinteracting with an interactive element on the modular tile 3006. Thevisual interface 3402 may include an interactive element 3406 to causethe visual interface 3402 to be hidden from display. The visualinterface 3402 may include an interactive element 3410, which may enableconfiguration of actions to be performed in response to interaction withthe modular tile 3006. Interaction with the interactive element 3410 toan “OFF” state may cause the visual interface 3402 to hide otherinteractive elements used to configure actions.

The display 3405 may be adjusted to the display 3415 when theinteractive element 3410 is adjusted to an “ON” state. In the “ON”state, the visual interface 3402 may include one or more interactiveelements to enable configuration of actions for one or more of themodular tiles 3002-3006. In the example shown, the visual interface 3402may enable actions to be configured for the modular tile 3006. Thevisual interface 3402 may include an interactive element 3420 (e.g.,“Scene(s)”). The interactive element 3420 may enable a user to select ascene to perform as an additional action upon interaction with themodular tile 3006. The scene may be associated with criteria forperforming actions. For example, the information 2900 may be stored by adevice to determined actions to perform based on a selected scene. Theinteractive element 3420 may include an interactive element 3430 toselect one of many scenes, which may have been pre-determined orspecified by a user. The interactive element 3420 may include a scrollbar 3440 to view other scenes not visible. The visual interface mayinclude an interactive element 3450 (e.g., “More”) to view additionalinteractive elements to configure actions. In some embodiments,selecting the interactive element 3450 may cause another graphicalinterface to be shown, such as the one shown in FIG. 35.

FIG. 35 shows an example of a graphical interface presented by a networkdevice according to some embodiments of the present disclosure.Specifically, FIG. 35 shows an example of a display 3505 presented by anaccess device according to some embodiments. The display 3500 maypresent one or more interactive components (e.g., elements interactiveareas, controls, or the like) to configure actions for a modular tilecorresponding to a network device. The display 3500 may include one ormore elements described earlier, such as interactive elements includedin FIG. 30. In some embodiments, the display 3500 may be presented inresponse to an interaction with any of the displays described withreference to FIGS. 30-34 or an interaction with a tile, such as any oneof the tiles 3002, 3004, 3006 described earlier. For example, thedisplay 3500 may be presented in response to an interaction with therules icon 3020, the more icon 3024, or any of the interactive elements3322, 3422, 3550 described earlier. Upon configuring actions via thedisplay 3500, actions may be performed in response to interaction withan interactive area to which the actions are assigned.

In the example shown in FIG. 35, the display 3500 includes one or moregraphical interfaces. The display 3500 may include one or moreinteractive elements. In some embodiments, the display 3500 may includean interactive element to configure one or more additional actions. Forexample, the display 3500 may include an interactive element 3510 toconfigure an action (e.g., “Action 1”) and an interactive element 3540to configure a secondary action (e.g., “Action 2”). An additional actionmay be configured to be performed based on one or more criteria.

In some embodiments, each of the interactive element 3510 and theinteractive element 3540 may control presentation of one or morecomponents (e.g., interactive elements or graphical interfaces) to bedisplayed. The components may enable configuration of criteria for oneor more additional actions. The components may include present one ormore options for selection of one or more criteria for the additionalactions. The options may include values, ranges, or any other criteriathat may be chosen. The components may be interactive to receive inputindicating criteria for the additional action.

In one example, interaction with the interactive element 3510 may enablecomponents such as an interactive element 3512, an interactive element3514, an interactive element 3516, an interactive element 3518, aninteractive element 3522, and an interactive element 3534 to bedisplayed. These interactive elements are illustrated as examples. Someembodiments may include more or fewer interactive elements, which may bedisplayed or arranged differently and which may be configureddifferently.

The interactive element 3512 may enable a user to configure of a timeperiod (e.g., “Activation Time”). The time period (e.g., 5 seconds) maybe when an action is triggered in response to interaction with aninteractive area. For example, as explained earlier, the time period maybe a threshold time period at or after which an action may be performed.

The interactive element 3514 may enable a user to configurenotification(s) (e.g., “Notification”). The interactive element 3514 mayenable a user to provide input for sending notifications such as a typeof notification, where to send a notification, and/or how to send anotification. Additional actions may include sending notifications basedon the input received

The interactive element 3516 may enable a user to input informationrelated to access control for performing actions. The interactiveelement 3516 may receive input about users that may be permitted toinitiate additional actions.

The interactive element 3518 may enable a user to input one or moremodes. Additional actions may be performed based on the chosen mode(s).In some embodiments, other criteria may be stored in association withthe mode(s).

The interactive element 3520 may enable a user to input one or morescenes. Additional actions may be performed based on the chosenscene(s). In some embodiments, other criteria may be stored inassociation with the scene(s).

The interactive element 3522 may enable a user to input schedules forperforming additional actions. The schedules may include a date and/ortime when additional actions are to be initiated.

The interactive element 3534 (e.g., “Device Settings/Attributes”). Theinteractive element 3534 may enable a user to indicate device settingsand/or attributes to be adjusted. Additional actions may includeadjusting network devices according to the device settings and/orattributes configured using the interactive element 3534. For example,the interactive element 3534 may be interactive to enable a user toindicate criteria, such as settings and attributes for a network deviceor group of network devices. The network device(s) may be adjusted tothe settings and/or attributes as an action. In some embodiments, theinteractive element 3534 may include a control 3526 to enable selectionof a network device or group of network devices to adjust. Theinteractive element 3534 may include a control 3528 to enable selectionof an attribute or a setting to adjust for a network device or group ofnetwork devices. The interactive element 3534 may include a control 3530to enable selection of criteria (e.g., a value or a state) to set for anattribute or a setting chosen using the control 3528. The interactiveelement 3534 may enable a user to configure an action for an adjustmentof multiple network devices or multiple groups of network devices basedon settings and/or attributes.

The display 3500 may include an interactive element 3550 (e.g., “More”)to cause the display 3500 to change to show additional interactiveelements when one or more interactive elements are unable to bedisplayed in the display 3500.

Now turning to FIGS. 36-38, flowcharts are illustrated of processes forinitiating actions using an interface configured to control operation ofa network device. FIG. 36 is an example of a process to initiate actionsfrom a graphical interface presented at a device (e.g., the accessdevice 108). FIG. 37 is an example of a process to initiate actions froma physical interface on a device (e.g., the device 422). The physicalinterface may correspond to a network device to be controlled. FIG. 38is an example of a process for initiating actions using an interfaceconfigured to control operation of a network device in accordance withsome embodiments.

In FIG. 36, a flowchart is illustrated of embodiments of a process 3600for initiating actions using an interface configured to controloperation of a network device in accordance with some embodiments.Specifically, the process 3600 provides techniques to enable one or moreactions (e.g., additional actions) to be performed in response tocontinued interaction with an interactive area of an interfacecorresponding to a network device. The one or more actions may includecontrolling operation of network devices connected to a network. Theaction(s) may include actions besides controlling operation of thenetwork device corresponding to the interactive area of an interface inwhich continued interaction occurs.

In some embodiments, the techniques to enable one or more actions inresponse to continued interaction with an interactive area may beimplemented by a computing device (e.g., the access device 108 ofFIG. 1) regardless of whether the computing device is connected to anetwork or in remote communication with the network. For example, theprocess 3600 may be performed by a computing device (e.g., the accessdevice 108 of FIG. 1) that is not connected to the network or not incommunication with the network (i.e., at a remote location outside thenetwork).

Process 3600 is illustrated as a logical flow diagram, the operation ofwhich represents operations that may be implemented in hardware,computer instructions, or a combination thereof. In the context ofcomputer instructions, the operations represent computer-executableinstructions stored on one or more computer-readable storage media that,when executed by one or more processors, perform the recited operations.Generally, computer-executable instructions include routines, programs,applications, objects, components, data structures, and the like thatperform particular functions or implement particular data types. Theorder in which the operations are described is not intended to beconstrued as a limitation, and any number of the described operationsmay be combined in any order and/or in parallel to implement theprocesses.

Additionally, the process 3600 may be performed under the control of oneor more computer systems configured with executable instructions and maybe implemented as code (e.g., executable instructions, one or morecomputer programs, or one or more applications) executing collectivelyon one or more processors, by hardware, or combinations thereof. Asnoted above, the code may be stored on a computer-readable storagemedium, for example, in the form of a computer program comprising aplurality of instructions executable by one or more processors. Thecomputer-readable storage medium may be non-transitory.

Beginning at block 3602, a network device (e.g., network device 302,network device 304, network device 306, or network device 308) may bedetermined to be connected to a network (e.g., the network 300).Determining that a network device is connected to a network may includeperforming operations to discover that the network device is on thenetwork. For example, block 3602 may be performed as part of discoveryprocess to identify one or more network devices connected to thenetwork. To determine that a network device is connected to a network,local storage of a computing device (e.g., access device 108) may besearched for a unique identifier associated with the network device. Aunique identifier may be associated with the network device when thenetwork device connects to the network. As such, locating a uniqueidentifier in storage may indicate that the network device is connectedto the network and thus, has been discovered. A unique identifier forthe network device may be received from the network device itself orfrom the network upon establishing connection with the network.

At block 3604, a graphical interface (e.g., any one of the displays2622, 3005, 3105, 3205, 3305, or 3505) may be generated. The graphicalinterface may include an interactive area corresponding to one or morenetwork devices connected to a network. The interactive area may beinteractive to control operation of the network device. At block 3606,the process 3600 may include displaying the graphical interfacegenerated at block 3604.

At block 3608, input corresponding to an interaction with theinteractive area may be received. The interactive area may correspond toone or more network devices connected to a network. The input receivedvia the interactive area may be associated with an adjustment forcontrolling operation of a network device connected to a network. Forexample, the access device 108 may receive, via the display 3005, inputcorresponding to interaction at area 3050 corresponding to the button3034. In some embodiments, the input may include informationcorresponding to an adjustment for an attribute or a setting related tooperation of the network device. For example, the input may includeinformation corresponding to a value to set or update an attribute or asetting related to operation of the network device. The input mayinclude data that may be used to determine information (e.g., anadjustment) to control operation of a network device.

In some embodiments, an adjustment to a network device may correspond toinput based on interaction with the interactive area. The input maycorrespond to or indicate a frequency of contact with the interactivearea, a distance over which the interaction occurred in the interactivearea, a duration (e.g., a time period) of the interaction with theinteractive area, type of contact with the interactive area, orcombinations thereof. A type of contact with the interactive area mayinclude physical contact with the interactive area by a user. Examplesof a physical contact may include, without limitation, touching,selecting, swiping, tapping, pressing, and/or any type of movement withthe interactive area. The physical contact with the interactive area mayinclude one or more contacts with the interactive area. In someembodiments, the input corresponding to interaction with an interactivearea may occur during a pre-determined time period.

At block 3610, a continued interaction with the interactive areaincluded in the graphical interface may be detected. The continuedinteraction may cause one or more actions to be performed at block 3612explained below. The interactive area may correspond to the networkdevice connected to a network. Continued interaction with theinteractive area may be detected by determining that the interactionwith the interactive area at block 3608 satisfies one or more criteria(e.g., a pre-determined time period) for the interactive area at block3608. The one or more criteria for continued interaction may be based ininteraction with an interactive area, such as a frequency of contactwith the interactive area, a distance over which the interactionoccurred in the interactive area, a duration (e.g., a time period) ofthe interaction with the interactive area, type of contact (e.g., aphysical contact) with the interactive area, or combinations thereof.

In one illustrative example, the continued interaction with theinteractive area may correspond to an interaction with the interactivearea exceeding a pre-determined time period. The interaction with theinteractive area may begin at block 3608. In some embodiments,continuous interaction (e.g., a physical contact) with the interactivearea is maintained during the interaction and the continued interaction.For example, a continued interaction may correspond to pressing thebutton 3034 at the area 3050 beyond a pre-determined time period.

At block 3612, one or more actions (e.g., an additional action) may beperformed. An action may be performed in response to detecting acontinued interaction with an interactive area described above withreference to block 3610.

In some embodiments, an action may include controlling operation of oneor more network devices. An action may be device specific such that theaction is performed for a type of network device. An operation of anetwork device may be controlled by transmitting an adjustment to thenetwork device. In some embodiments, performing an action may includetransmitting a request including an adjustment for controlling operationof one or more network devices connected to a network. The adjustmentmay be specific to a type of network device controlled by the action.Examples of an action may include, without limitation, initiating ascene for operation of network devices or initiating a mode foroperation (e.g., home modes, away modes, or sleep modes) of one or morenetwork devices. A network device controlled in response to a continuedinteraction with the interactive area may correspond to a network devicethat is different than the one corresponding to the interactive area.For example, a continued interaction with an interactive areacorresponding to a network device may cause an action to be performedfor a different network device.

In some embodiments, an action may be performed based on a rule. Therule may indicate one or more actions to perform for controllingoperation of a network device. The rule may be related to operation ofone or more other network devices connected to the network. The rule maybe based on a time, a date, an event, a condition related to operationof a network device, or combinations thereof. In some embodiments, theaction may be performed according to a schedule (e.g., a scheduledaction). In some embodiments, the rule to perform an action may bemodified or created based on one or more criteria received via agraphical interface.

In some embodiments, an action may include sending a notification. Thenotification may be send to a network to which network devices areconnected. The notification may be related to operation of a networkdevice (e.g., the network device determined at 3602). For example, thenotification may indicate an adjustment to a network device based on anadjustment corresponding to the input received at 3608. The notificationmay include an electronic message (e.g., an email message or a textmessage).

In some embodiments, a computing device may store information indicatingone or more actions that can be performed in response to continuedinteraction with an interactive area. For example, as described withreference to FIG. 3, the access device 108 may store information incache. The information may include information stored on a computingdevice, such as information 2800 of FIG. 28 or information 2900 of FIG.29.

At block 3614, an adjustment may be transmitted. The adjustment may bebased on the input received via interaction with an interactive areacorresponding to a network device. The adjustment may be transmitted toa network to which a network device is connected. The network device maycorrespond to the interactive area from which input was received for anadjustment. Operation of the network device may be controlled using thetransmitted adjustment.

In some embodiments, the adjustment may be transmitted in an updatedstatus for one or more network devices connected to the network. Theupdated status may be transmitted to the network via the cloud network114 if the network device transmitting the updated status is not locatedin the network. The transmitted adjustment may be received by a networkdevice to which the adjustment is to be made. In some embodiments, whenthe network device 306 is not powered to an “on” state, the networkdevice 306 may receive the transmitted adjustment upon reconnecting tothe network after switching to an “on” state. At block 3616, the process3600 may end.

It will be appreciated that process 3600 is illustrative and thatvariations and modifications are possible. Steps described as sequentialmay be executed in parallel, order of steps may be varied, and steps maybe modified, combined, added or omitted. For example, the adjustmenttransmitted at block 3614 may be performed before, concurrently with, orafter an action is performed at block 3612. Process 3600 may beperformed for different network devices that are connected to a network.It should be noted that process 3600 may be performed concurrently eachof the network devices that are connected to the network. Process 3600may be performed for any number of network devices that can becontrolled via a graphical interface displayable by a computing device.Further, the process 3600 may be performed for several actions that maybe performed in response to continued interaction with an interactivearea corresponding to a network device.

FIG. 37 is a flowchart illustrating embodiments of a process 3700 forinitiating actions using an interface configured to control operation ofa network device in accordance with some embodiments. Specifically, theprocess 3700 provides techniques to enable one or more actions to beperformed in response to continued interaction over a time period withan interactive area corresponding to a network device. The techniquesmay be implemented by a computing device (e.g., the computing device2722 or any of the network devices 2702, 2704, 2706, 2708) regardless ofwhether the computing device is connected to a network to which networkdevices are connected or whether the computing device is incommunication with the network via a remote connection (e.g., via cloud114). The actions performed in response to continued interaction mayinclude controlling operation of different network devices connected toa network.

Process 3700 is illustrated as a logical flow diagram, the operation ofwhich represents operations that may be implemented in hardware,computer instructions, or a combination thereof. In the context ofcomputer instructions, the operations represent computer-executableinstructions stored on one or more computer-readable storage media that,when executed by one or more processors, perform the recited operations.Generally, computer-executable instructions include routines, programs,applications, objects, components, data structures, and the like thatperform particular functions or implement particular data types. Theorder in which the operations are described is not intended to beconstrued as a limitation, and any number of the described operationsmay be combined in any order and/or in parallel to implement theprocesses.

Additionally, the process 3700 may be performed under the control of oneor more computer systems configured with executable instructions and maybe implemented as code (e.g., executable instructions, one or morecomputer programs, or one or more applications) executing collectivelyon one or more processors, by hardware, or combinations thereof. Asnoted above, the code may be stored on a computer-readable storagemedium, for example, in the form of a computer program comprising aplurality of instructions executable by one or more processors. Thecomputer-readable storage medium may be non-transitory.

Beginning at block 3702, input corresponding to an interaction with aninteractive area on an interface (e.g., a physical interface or agraphical interface) may be received. The interface may correspond to orbe associated with a network device. For example, the device 2722 mayreceive input corresponding to an interaction with an interactive areaon the interface 2710. The interface 2710 may be configured to controlthe network device 2702. The input may correspond to an action forcontrolling operation of the network device.

As explained above, input corresponding to an interaction with aninteractive area may correspond to or indicate a frequency of contactwith the interactive area, a distance over which the interactionoccurred in the interactive area, a duration (e.g., a time period) ofthe interaction with the interactive area, type of contact with theinteractive area, or combinations thereof. A type of contact with theinteractive area may include physical contact with the interactive areaby a user.

At block 3704, the input may be used to determine the action forcontrolling operation of the network device. Continuing with the exampleabove, the device 2722 may process the input to determine an action forcontrolling operation of the network device 2702 corresponding to theinterface 2710. In some embodiments, the device 2722 may use informationstored in the cache 2730, e.g., the information 2800 or the information2900, to determine an action to perform to control the network device2702. Using the information, the action may be determined based on thetype of input, a frequency of the input, a duration of the input, orother rules based on input. For example, the device 2722 may determinethat the action for controlling the network device 2702 includeschanging a power state of the network device 2702. In some embodiments,the action may be determined based on information stored by anothernetwork device connected to the network, such as information stored byone of network devices 2702, 2704, 2706, 2708.

In some embodiments, an action may include controlling operation of oneor more network devices. An action may be device specific such that theaction is performed for a type of network device. An operation of anetwork device may be controlled by transmitting an adjustment to thenetwork device. In some embodiments, performing an action may includetransmitting a request including an adjustment for controlling operationof one or more network devices connected to a network. The adjustmentmay be specific to a type of network device controlled by the action.Examples of an action may include, without limitation, initiating ascene for operation of network devices or initiating a mode foroperation (e.g., home modes, away modes, or sleep modes) of one or morenetwork devices.

In some embodiments, an action may be performed based on a rule. Therule may indicate one or more actions to perform for controllingoperation of a network device. The rule may be related to operation ofone or more other network devices connected to the network. The rule maybe based on a time, a date, an event, a condition related to operationof a network device, or combinations thereof. In some embodiments, theaction may be performed according to a schedule (e.g., a scheduledaction). In some embodiments, the rule to perform an action may bemodified or created based on one or more criteria received via agraphical interface.

In some embodiments, an action may include sending a notification. Thenotification may be send to a network to which network devices areconnected. The notification may be related to operation of a networkdevice (e.g., the network device determined at 3602). For example, thenotification may indicate an adjustment to a network device based on anadjustment corresponding to the input received at 3608.

At block 3706, data may be transmitted to indicate the action forcontrolling operation of the network device. For example, uponprocessing the input to determine the action for controlling operationof the network device 2706, the device 2722 may transmit data indicatingthe action(s) determined based on the input at 3704. Transmitting thedata may cause the action to be performed. For example, the networkdevice 2706 may perform the action (e.g., change a power state) based onreceive instructions indicated in the transmitted data.

At block 3708, a determination may be made that the interaction with theinteractive area exceeds a threshold time period. For example, thedevice 2722 may determine whether a user interacts (e.g., contacts) aninteractive area on the interface 2710 for more than a threshold timeperiod (e.g., the activation time 2830). The threshold time period maybe associated with one or more actions 2820 to be initiated uponexceeding the threshold time period.

At block 3710, an action may be performed when the threshold time periodis exceeded. Performing the action may include transmitting data to thenetwork device. The data may indicate the action(s) to perform and/orother criteria related to performing those actions. The data mayindicate adjustments to attributes and/or settings (e.g., a power state)to be changed. An action may be performed based on the transmitted data.

It will be appreciated that process 3700 is illustrative and thatvariations and modifications are possible. Steps described as sequentialmay be executed in parallel, order of steps may be varied, and steps maybe modified, combined, added or omitted. Process 3700 may be performedfor each network device that is determined to be connected to thenetwork. It should be noted that process 3700 may be performedconcurrently for other network devices that are determined to beconnected to the network. Process 3700 may be performed for any numberof network devices that can be controlled via a graphical interfacedisplayable by a computing device. Further, the process 3700 may beperformed for several actions that may be performed in response tocontinued interaction with an interactive area corresponding to anetwork device.

FIG. 38 is a flowchart illustrating embodiments of a process 3800 forinitiating actions using an interface configured to control operation ofa network device in accordance with some embodiments. Specifically, theprocess 3800 provides techniques to enable one or more actions to beperformed in response to continued interaction over a time period withan interactive area corresponding to a network device. The techniquesmay be implemented by a computing device (e.g., the computing device2722 or any of the network devices 2702, 2704, 2706, 2708) regardless ofwhether the computing device is connected to a network to which networkdevices are connected or is in remote communication with the network.The actions performed in response to continued interaction may includecontrolling operation of different network devices connected to anetwork.

Process 3800 is illustrated as a logical flow diagram, the operation ofwhich represents operations that may be implemented in hardware,computer instructions, or a combination thereof. In the context ofcomputer instructions, the operations represent computer-executableinstructions stored on one or more computer-readable storage media that,when executed by one or more processors, perform the recited operations.Generally, computer-executable instructions include routines, programs,applications, objects, components, data structures, and the like thatperform particular functions or implement particular data types. Theorder in which the operations are described is not intended to beconstrued as a limitation, and any number of the described operationsmay be combined in any order and/or in parallel to implement theprocesses.

Additionally, the process 3800 may be performed under the control of oneor more computer systems configured with executable instructions and maybe implemented as code (e.g., executable instructions, one or morecomputer programs, or one or more applications) executing collectivelyon one or more processors, by hardware, or combinations thereof. Asnoted above, the code may be stored on a computer-readable storagemedium, for example, in the form of a computer program comprising aplurality of instructions executable by one or more processors. Thecomputer-readable storage medium may be non-transitory.

Beginning at block 3802, a network device (e.g., the network device2702, the network device 2704, the network device 2706, or the networkdevice 2708) may be determined to be connected to a network (e.g., thenetwork 2700). Determining that a device is connected to a network mayinclude performing operations to discover that the device on thenetwork.

At block 3804, data determined at an access device may be received. Forexample, in FIG. 27B, device 2722 may receive data determined at accessdevice 108. In another example, network device 2702 may receive datadetermined at access device 108. The data may include data correspondingto information 2740 stored in cache 2602. As explained above,information 2740 may be determined using access device 108. For example,data corresponding to information 2740 may be determined based on inputcorresponding to interaction with an interactive area of a graphicalinterface displayed by access device 108. The data received from anaccess device may indicate an adjustment corresponding to each of one ormore attributes or one or more settings related to operation of anetwork device (e.g., network device 2708). The received data mayindicate a threshold time period associated with an adjustmentcorresponding to an attribute or a setting for a network device.

At block 3806, input may be detected corresponding to interaction withan interactive area of an interface corresponding to the network device.The interaction may be performed by movement in the interactive area.For example, the interaction may be performed by one or more contactswith the interactive area. The interactive area may correspond to one ormore network devices connected to a network. The input may be detectedbased on data received at a device from the interface. The detectedinput may be associated with an adjustment for controlling operation ofa network device connected to a network. In one example, the input maybe detected corresponding to interaction with an interactive area ofinterface 410. In another example, the input may be detectedcorresponding to interaction at area 3050 of interface 3005.

In some embodiments, the detected input may include informationcorresponding to an adjustment for an attribute or a setting related tooperation of the network device. For example, the input may includeinformation corresponding to a value to set or update an attribute or asetting related to operation of the network device. The input mayinclude data that may be used to determine information (e.g., anadjustment) to control operation of a network device.

At block 3808, using the detected input, a determination is made as towhether the interaction with the interactive area of the interfaceexceeds a threshold time period. In some embodiments, the interactionexceeds a threshold time period when the interaction with theinteractive area is continuous and occurs for a duration of thethreshold time period.

At block 3810, information indicating an adjustment may be sent. Theadjustment may correspond to an adjustment of an attribute related tooperation of a network device corresponding to the interactive area. Theinformation may be sent when an interaction with the interactive areaexceeds the threshold time period associated with the adjustment. Thesent information may cause operation of the network device to becontrolled using the adjustment. The information may be sent bytransmitting the information. In some embodiments, the information maybe sent by storing the information into memory. Process 3800 may end at3812.

In some embodiments, process 3800 may include determining a statusrelated to operation of the network device using the adjustment. Forexample, device 2722 may determine a status related to operation ofnetwork device 2708. A message indicating that the status may be sent.For example, the message may be sent from device 2722 to access device108. The sent message may cause a modification to a graphical interfacedisplayed by the access device. The modification may include displayingthe status.

In some embodiments, process 3800 may include generating and displayinga graphical interface. The graphical interface includes an interactivearea to configure operation of one or more network devices connected tothe network. Process 3800 may include determining additional data basedon input corresponding to an interaction with the interactive areaincluded in the graphical interface. The additional data may indicate amodification to the threshold time period associated with theadjustment. In some embodiments, process 3800 may include determining astatus related to operation of the network device using the adjustment.The graphical interface may be modified to indicate that the determinedstatus.

It will be appreciated that process 3800 is illustrative and thatvariations and modifications are possible. Steps described as sequentialmay be executed in parallel, order of steps may be varied, and steps maybe modified, combined, added or omitted. Process 3800 may be performedfor each network device that is determined to be connected to thenetwork. It should be noted that process 3800 may be performedconcurrently for other network devices that are determined to beconnected to the network. Process 3800 may be performed for any numberof network devices that can be controlled via a graphical interfacedisplayable by a computing device. Further, the process 3800 may beperformed for several actions that may be performed in response tocontinued interaction with an interactive area corresponding to anetwork device.

FIG. 39 illustrates an example of a network 3900. Similar to the localarea network 100 described above, the network 3900 includes networkdevices 3902, 3904, 3906, access devices 3908, 3912, a gateway 3910, andan external network 3914. Also illustrated in FIG. 39 is an accessdevice 3922 that can connect to devices on the network 3900 from aremote location that is outside of the range of signals provided by thegateway 3910. The gateway 3910, the network devices 3902, 3904, 3906,the access devices 3908, 3912, 3922, and the external network 3914 maybe similar and have the same functionalities and capabilities as thegateway 110 or 112, the network devices 102, 104, or 106, the accessdevice 108, and the external network 114 described above with respect toFIG. 1. It should be appreciated that the network 3900 may have othercomponents than those depicted. Further, the embodiment shown in thefigure is only one example of a network that may incorporate anembodiment of the invention. In some other embodiments, network 3900 mayhave more or fewer components than shown in the figure, may combine twoor more components, or may have a different configuration or arrangementof components. For example, while a certain number of network devices,access devices, and gateways are shown in FIG. 39, one of ordinary skillin the art will appreciate that any number of network devices, accessdevices, and gateways may be included in the network 3900.

Network device 3906 may include an automation switch that a user canremotely access, configure, and control. The network device 3906 alsoallows a user to remotely access, configure, and control any electronicdevice that can be plugged into the network device 3906. Network devices3902 and 3904 may include any type of network device that a user canremotely access, configure, and control, such as a network-connectedtelevision, a network-connected video game console, a network-connectedvideo camera, a network-connected appliance, a network-connected light,or any other network-connected device.

Access devices 3908, 3912, and 3922 may include any type of accessdevice that can be used to access devices on the network 3900, includingone or more of the network devices 3902, 3904, 3906. For example,similar to the access device 108 described above, access devices 3908,3912, and 3922 may include any human-to-machine interface with networkconnection capability that allows access to the network 3900.

In some embodiments, the network 3900 may include a local area networkin which the network devices 3902, 3904, 3906 and the access devices3908, 3912 communicate with one another through the gateway 3910 usingwired and wireless connections (e.g., using WiFi™ communicationprotocols) or communicate with one another directly using one or moredirect communication protocols (e.g., UPnP, Bluetooth™, Zigbee™, UWB,WiFi-Direct, BLE, sound frequencies, IR frequencies, and/or the like).Secure connections may be established between the network devices 3902,3904, 3906 and the one or more service providers through externalnetwork 3914. Secure connections may also be established between theaccess devices 3908, 3912 and one or more service providers throughexternal network 3914. Network address translation (NAT) Traversalapplications may be used by networks of the service providers toestablish the secure connections. In some examples, the secureconnections may include secure TCP connections.

The gateway 3910 may provide NAT services for the network devices 3902,3904, 3906 and access devices 3908, 3912. The gateway 3910 may performvarious types of NAT, including static NAT, dynamic NAT, overloading,overlapping, or any other available NAT service. For example, thegateway 3910 may perform multiplexing to maintain the different secureconnections using different TCP or user datagram protocol (UDP) ports.Communications transmitted by the network devices 3902, 3904, 3906 andthe access devices 3908, 3912 may include IP packets. One of ordinaryskill in the art will appreciate that the communications may includeother types of packets. Each IP packet may include a source address, asource port, a destination address, and a destination port. The sourceaddress includes the private IP address of the device from which thecommunication originated. The source port includes the TCP or UDP portnumber assigned to the packet by the originating device. The destinationaddress includes the IP address of the device to which the communicationis destined to be received. The destination port includes the TCP or UDPnumber that the originating device is requesting be opened. Uponreceiving a packet, the gateway 3910 may map private IP addresses ofeach of the network devices 3902, 3904, 3906 and the access devices3908, 3912 to one or more public IP addresses maintained by the gateway3910. The one or more public IP addresses may be assigned to the gateway3910 (e.g., by an administrator or assignor of IP addresses) and may beunique to that gateway 3910. The gateway 3910 may also map the ports ofeach of the network devices 3902, 3904, 3906 and the access devices3908, 3912 to TCP or UDP ports of the gateway 3910.

As illustrated in FIG. 39, multiple client devices (network devices3902, 3904, 3906 and the access devices 3908, 3912) are connected to thegateway 3910 in order to access network connectivity. A gateway deviceinterface may be provided that allows a user to control and/or configurethe client devices.

FIG. 40 illustrates an embodiment of a system 4000 for detecting networkdevices (or other client devices) connected to the gateway 3910, andobtaining meta-information and control capabilities of the networkdevices. The meta-information and the control capabilities may be usedto allow users to control the connected devices using the gateway deviceinterface. Gateway 3910 is connected to cloud network 3914, and allowsnetwork device 3902 to connect to the cloud network 3914, the Internet,or other external networks using the gateway 3910. In some embodiments,the network device 3902 may be a home automation network device thatallows a user to access, monitor, control, and/or configure various homeappliances located within the user's home, such as a television, radio,light, microwave, iron, space heater, sensor, and/or the like. In someembodiments, the network device 3902 may include a network-connectabledevice, such as a smart television that can connect to the Internet, anetwork-connectable appliance, or any other device that can connect to acommunications network.

In an embodiment, a gateway device application may be run on an accessdevice 3908. The application may include a mobile application (e.g., anIoS™ application, an Android™ application, or any other appropriatemobile application). The access device 3908 may execute the applicationto cause the access device 3908 to present a gateway device interface4022 displaying information and control objects for each deviceconnected to the gateway 3910. In some embodiments, the access device3908 may execute a web page that presents the gateway device interface4022. A user may log in to an account with a user name and a password inorder to access the gateway device interface 4022 that is specific tothe user's gateway 3910.

In some embodiments, the gateway 3910 can maintain or access a devicedirectory 4002. In some embodiments, the device directory 4002 can be alocal cache located in onboard storage of the gateway 3910. In suchembodiments, the local device directory 4002 may be associated with thegateway device application executed by or run on access device 3908, andmay be accessed only by the gateway device application so that otherapplications running on the access device 3908 cannot access the devicedirectory 4002. In some embodiments, the device directory 4002 can be aremote storage, database, cache, or other storage mechanism maintainedby a web server that provides access to the web-based gateway deviceinterface web page.

The device directory 4002 can contain a list of devices that are knownby the gateway 3910 to exist on the local area network (e.g., devicesthat are connected or that have been connected to the gateway 3910). Forexample, the device directory 4002 may include a separate record foreach known device. The record for a device may include meta-informationand control capabilities. The information for a device record may beused to display data related to that device to a user in the gatewaydevice interface. Meta-information for a device may include identifyinginformation of the device, a device specific interface object moduleindicating a device specific interface object for the device, usagestatistics of the device, activity type, status or state of the device,or any other appropriate information that is exposed or published to thegateway device. For example, a connected device may expose identifyinginformation including a unique identifier of the device, such as auniversally unique identifier assigned by a cloud server, a MAC address,a serial number of the device, or any other identifier unique to thedevice. As another example, a connected device may expose a name of itsmanufacturer. As described in more detail below, the gateway 3910 candetermine a device specific interface object for a device using themeta-information exposed by the device. A device specific interfaceobject may include an icon that readily identifies the specific deviceto a user in the gateway device interface.

In some embodiments, when the gateway device application or web page forthe gateway device interface 4022 is run on the access device 3908, theaccess device 3908 can access information from the device directory 4002to present the display 4022 with device-specific information. In someembodiments, the gateway 3910 can send the information contained withinthe device directory 4002 to the access device 3908. In someembodiments, when a web-based gateway graphical interface is used, a webserver may send the information contained within the device directory4002 to the access device 3908. In some embodiments, the access device3908 may store a local copy of the device directory 4002. The accessdevice 3908 may access information from the local copy of the devicedirectory 4002. In any of these embodiments, the gateway deviceapplication or web page can obtain the information in the devicedirectory 4002 in order to display the device-specific information. Inan example, the display 3922 can include a displayed tile (e.g., tile4102A displayed in FIG. 41) for each connected network device having arecord in the device directory 4002.

Exemplary communications are shown in FIG. 40 that are used to determinethe information used to populate the device directory 4002. For example,the gateway 3910 may discover or detect that the network device 3902 iswithin communication signal range of the gateway 3910. Upon detectingthe presence of the network device 3902, the gateway 3910 may send acommunication 4010 to the network device 3902. The communication 4010may include a request to connect and a query for meta-information and/orcontrol capabilities. In some embodiments, a request to connect and aquery for meta-information and/or control capabilities may be sent inseparate communications. For example, the query may be sent after thenetwork device 3902 is connected to the gateway 3910. In someembodiments, the network device 3902 may initiate the discovery bysending a discovery message (not shown) to the gateway 3910. Thediscovery of network device 3902 may be initiated when the networkdevice 3902 is rebooted (e.g., powered on, reset or restored to defaultsettings, or the like). For example, when the network device 3902 isrebooted, it may broadcast one or more discovery messages on the localarea network to discover gateway devices in the local area network. Inone example, a discovery message may be broadcast during a discoveryprocess according to a universal plug and play (UPnP) protocol, aZigbee™ protocol, a Bluetooth™ protocol, or any other appropriateprotocol. In response to the discovery message, the network device 3902may receive a communications from gateway 3910 indicating that thegateway 3910 is located within the local area network. The communicationindicating presence of the gateway 3910 may be part of communication4010 or may be a separate communication sent prior to communication4010.

After receiving communication 4010 requesting meta-information and/orcontrol capabilities, the network device 3902 may send communication4012 to the gateway 3910. The communication 4012 includes themeta-information and the control capabilities that are exposed by thenetwork device 3902. The gateway 3910 may populate the device directory4002 record for the network device 3902 with the meta-information andthe control capabilities, and also with further information determinedusing the meta-information and the control capabilities. For example,the meta-information includes identifying information of the networkdevice 3902 that is exposed or published for access by the gatewaydevice 3910. For example, the network device 3902 may expose a uniqueidentifier of the device, such as a universally unique identifier, a MACaddress, a serial number, or any other identifier unique to the networkdevice 3902. As another example, the network device 3902 may expose aname of its manufacturer. In some embodiments, the particularmeta-information and control capabilities that are exposed by thenetwork device 3902 may be determined by the manufacturer of the networkdevice 3902. For example, the manufacturer of the network device 3902may agree with a manufacturer of the gateway 3910 to allow its networkdevices to expose their MAC addresses and the name of the manufacturer.The network device 3902 manufacturer may also allow its network devicesto expose their control application programming interface (API) to allowthe gateway 3910 to know how to control the network device 3902. In someembodiments, a user may specify the level of detail exposed in themeta-information and/or control information.

The gateway 3910 can determine a device specific interface object forthe network device 3902 depending on the information included in themeta-information exposed by the network device 3902. A device specificinterface object may include an icon that identifies the specific deviceto a user in the gateway device interface 4022. The device specificinterface object may be displayed in the network device's 3902 tile inthe gateway device interface 4022 (e.g., the icon 4104A displayed inFIG. 41). For example, an access device, or other device, may execute adevice specific interface object module to display a particular devicespecific interface object that is specific to the network device 3902.In some cases, the device specific interface object module may includeinstructions or code that can be executed by a processor to cause thedevice specific interface object to be displayed. Depending on thedetail of the meta-information, the device specific interface object maybe a generic icon, a default icon, or an icon that is custom andspecific to the network device 3902.

For example, the meta-information may include the device specificinterface object for the network device 3902. In one example illustratedin FIG. 41, the network device 3902 may include a home automation switchthat a user can remotely access, configure, and control, and that alsoallows a user to remotely access, configure, and control any electronicdevice that can be plugged into the network device 3902. The homeautomation switch is displayed in tile 4102A of the gateway deviceinterface 4022 provided in the graphical interface display 4100. Thenetwork device 3902 may store its own device specific interface objectmodule, and may send it to the gateway 3910 with the meta-information.The gateway 3910 can then utilize the received device specific interfaceobject module for generating the gateway device interface 4022. Forexample, the gateway 3910 may transmit the device specific interfaceobject module to the gateway device application running on the accessdevice 3908. The access device 3908 may execute the device specificinterface object module (e.g., using a processor) to generate anddisplay the device specific interface object (e.g., icon 4104A in FIG.41) for identifying the home automation switch and conveying whichnetwork device is being viewed in the gateway device interface 4022. Insuch an example, the device specific interface object may be custom tothe network device 3902 or may be a default device specific interfaceobject that is specific to the type of network device 3902 (e.g., adefault icon for home automation switches made by a particularmanufacturer). The custom or default device specific interface objectmay readily identify to the user that the network device 3902 that isbeing controlled or viewed in the gateway device interface 4022 is ahome automation switch.

In another example, the meta-information may include identifyinginformation of the network device 3902, but may not include an actualdevice specific interface object module. For example, the identifyinginformation may include a universally unique identifier, a MAC address,a serial number, make and model of the network device 3902, or otheridentifying information. The gateway 3910 may use the identifyinginformation to determine a default device specific interface object forthe type of network device 3902. For example, the gateway 3910 may sendthe identifying information to a server in the cloud network 3914. Inone example, the gateway 3910 may send communication 4014 with theidentifying information including the unique identifier. The cloudnetwork 3914 may be maintained by a manufacturer of the gateway 3910, amanufacturer of the network device 3902, a web server, or other externalnetwork provider.

In one example, the cloud network 3914 may be maintained by the gateway3910 manufacturer. In such an example, the network device 3902 may beregistered with the cloud network 3914. For example, the network device3902 may send information to the cloud network 3914 using communication4006. The cloud network 3914 server may send information back to networkdevice 3902, such as a confirmation message. The cloud network 3914 maystore information related to the network device 3902 in a database 4004.For example, a record may be kept for the network device 3902 thatincludes information such as device type, make and model, or otherdetails related to the network device 3902. In some embodiments, thecloud network 4004 may store the device specific interface object modulecorresponding to the device specific interface object of the networkdevice 3902. The cloud network 4004 may store the device specificinterface object module in the network device's 3902 record stored inthe database 4004. The record may also include other information, suchas one or more security keys associated with the network device 3902.The record may be identified using the unique identifier of the networkdevice 3902. The cloud network 3914 server may then send communication4016 to gateway 3910 with the relevant information from the recordidentifying the type of the network device 3902. The gateway 3910 maythen determine a device specific interface object for the network device3910 using the information identifying the type of the network device3902. For example, the communication 4016 may include an indication thatthe network device 3902 is a smart television. The gateway 3910 mayretrieve a default icon for smart televisions, and may assign thedefault icon to the record associated with the network device 3902 inthe device directory 4002. The default icon may then become the devicespecific interface object for the network device 3902. In one example,the default icon for the network device 3902 may include icon 4104Billustrated in FIG. 41.

In another example, the cloud network 3914 may be maintained by amanufacturer of the network device 3902. For example, gateway 3910 maysend the unique identifier to the cloud network 3914 of the networkdevice 3902 manufacturer. The manufacturer may maintain database 4004that includes records for devices and their identifiers (e.g., MACaddresses, serial numbers, or the like). The record may indicate thetypes of the devices. The manufacturer may look up the type of thenetwork device 3902 in the database 4004, and may return the informationidentifying the type of the network device 3902 to the gateway 3910. Thegateway 3910 may then determine a device specific interface object forthe network device 3910 using the received information. For example, thecommunication 4016 may include an indication that the network device3902 is a smart television. The gateway 3910 may retrieve a default iconthat is stored for smart televisions, and may assign the default icon tothe record associated with the network device 3902 in the devicedirectory 4002. The default icon may then become the device specificinterface object for the network device 3902. In one example, thedefault icon for the network device 3902 may include icon 4104Billustrated in FIG. 41.

In another example, the meta-information may include the type of thenetwork device 3902. For example, the meta-information may indicate tothe gateway 3910 that the network device 3902 is a smart television,similar to the examples above. The gateway 3910 may retrieve a defaultdevice specific interface object or icon that is stored for smarttelevisions, and may assign the default icon to the record associatedwith the network device 3902 in the device directory 4002.

In some embodiments, the gateway 3910 may monitor one or more behaviorcharacteristics of the network device 3902 to determine a type of thenetwork device 3902. For example, the meta-information may not includeany information that can be used to identify the specific network device3902 or the type of the network device 3902. The gateway 3910 is in aposition to monitor communications of the devices since it is thegateway for network communications for all connected devices. In oneexample, the network device 3902 is a smart television. The gateway 3910may monitor communications to and from the network device 3902 to detecta type, amount, or other characteristic of the communications. Forexample, the gateway 3910 may determine from one or more communicationpackets transmitted to the network device 3902 that the network device3902 is receiving video data. The gateway 3910 may also determine thatthe network device 3902, over a period of time (e.g., an hour, a day, aweek, or other appropriate amount of time) does not receive voice data.The gateway 3910 may determine a type of the network device based on themonitored behavior characteristics. For example, based on the networkdevice 3902 receiving video data and no voice data, the gateway 3910 maydetermine that the network device 3902 is a smart television. Thegateway 3910 may then retrieve a default icon that is stored for smarttelevisions, and may assign the default icon to the record associatedwith the network device 3902 in the device directory 4002. In someembodiments, the gateway 3910 may prompt a user to confirm the type ofdevice using the gateway device interface. For example, the gatewaydevice interface may display a notification with an option to confirmthe determined type.

In some embodiments, the monitored behavior characteristics may also bestored in the device directory 4002. For example, monitored behaviorcharacteristics, such as bandwidth usage, network usage time, powerlevel, or any other behavior may be stored in the network device's 3902record. For example, the behavior characteristics may be stored in ausage statistics section of the record. The usage statistics may bedisplayed to a user in the gateway device interface (e.g., usagestatistics 4108B or 4108E).

In some embodiments, the gateway 3910 may be unable to determine a typeof the network device 3902. The gateway 3910 may use a generic icon inthe event a default or custom icon cannot be assigned to the networkdevice 3902. In some embodiments, a user may select a customized iconfor use as the device specific interface object. For example, thegateway device interface may offer a list of selectable icons related tothe network device 3902. In another example, the user may upload apicture or other graphic to be used as the device specific interfaceobject.

As previously discussed, the network device 3902 may also send itscontrol capabilities to the gateway 3910. The gateway 3910 can determineone or more primary control capabilities corresponding to the networkdevice 3902 based on the control capabilities. For example, the gateway3910 may determine that the primary control capabilities of the networkdevice 3902 include turning the network device 3902 on and off. Thegateway 3910 may determine the primary control capabilities based on thefunctions that the gateway 3910 has access to. For example, the networkdevice 3902 may expose less than all of its control capabilities to thegateway 3910 (e.g., only a portion of an API used to control the networkdevice 3902). The gateway device 3910 may transmit the primary controlcapabilities to the access device 3908. The application running thegateway device interface may use the primary control capabilities togenerate and display one or more control objects. As explained withreference to FIG. 41 below, a control object allows a user to controlthe connected device using the one or more primary control capabilities.The primary control capabilities are a limited set of controlcapabilities as compared to a full set of control capabilities that areprovided in a proprietary interface of the connected device. Forexample, the primary control capabilities that are facilitated using oneor more control objects in the gateway device interface may includeturning a network device on or off, whereas a full set of controloptions provided in a proprietary interface specific to the networkdevice may include an ability to adjust settings, configurations, orother functionalities of the device. An example of a proprietaryinterface is illustrated in FIGS. 42-44.

In some embodiments, the gateway 3910 can transmit a resource bundlecorresponding to network device 3902 to the access device 3908. Theresource bundle may include the device specific interface object moduleand the primary control capabilities or control objects. The resourcebundle can be embodied as a structured folder structure whose contentsdefine all visual and interactive elements in a tile for the networkdevice 3902. The resource bundle may include a unique device ID andfiles defining graphical content to be displayed in the gateway deviceinterface. The graphical content can include definitions of interactivecontrol elements for the interface module. The resource bundle caninclude templates defining interactive control states for each of theinteractive control elements or interface objects, language translationsfor tile text, any menus for the tile, and graphical content of themenus. In some examples, the resource bundle can define templates, text,and graphical content using a markup language, such as XML, HTML, or anyother appropriate markup language.

The gateway 3910 may also send an indicator representing a state ofnetwork device 3902 to the access device 3908. In embodiments, thecommunication 4012 or other communication from network device 3902 caninclude a last known state of the network device 3902 and/or historicaldata associated with the network device 3902. The graphical interfacedisplay of the gateway device interface 4022 running on the accessdevice 3908 can then reflect a current state and historical data for thenetwork device 3902 using the gateway device interface. In one exampleusing the state information, an interface object or tile for the networkdevice 3902 displayed in the graphical interface display of the gatewaydevice interface 4022 can indicate an ‘on’ or ‘off’ state for thenetwork device 102 when the network device is powered on or off.

In some embodiments, the gateway 3910 can send the meta-information ofthe network device 3902 to other network devices connected to thegateway 3910. The other network devices can utilize themeta-information, for example, to display information related to thenetwork device 3902 in graphical interfaces associated with the devices.For example, a network printer may receive meta-information related to anetwork-connected camera. A graphical interface used to display printerfeatures and functionality may use the meta-information to displayinformation related to the network-connected camera.

Various graphical interface displays are discussed below with respect toFIGS. 4-7. In some embodiments, the graphical interface displays 4100,4200, 4300, and 4400 discussed below are displayed on an access device.For ease of explanation, the monitoring and control operations discussedwith reference to FIGS. 41-44 are described in the context of anapplication and/or a gateway device interface executing on an accessdevice with a touchscreen display device. However, the operations arenot intended to be limited to an access device. It is to be understoodthat the graphical interface displays 4100, 4200, 4300, and 4400illustrated in the exemplary embodiments of FIGS. 41-44 can be readilyadapted to be rendered on displays of a variety of computing deviceplatforms running a variety of operating systems.

FIG. 41 illustrates an example of a graphical interface display 4100 ofthe gateway device interface that can be implemented by an access device(e.g., through execution of a mobile application, a web-basedapplication, or any other appropriate program) to allow a user to viewinformation related to and to control different devices that areconnected to a gateway device. The graphical interface display 4100illustrated in FIG. 41 includes tiles 4102A, 4102B, 4102C, 4102D, and4102E that each display information and/or interactive control objectsfor various devices connected to a gateway device (e.g., gateway device3910). A user may interact with the connected devices through thegateway device interface 4022 using the primary control capabilities ofthe connected devices. The interactions can include monitoring theconnected devices, controlling the connected devices, or otherwiseinteracting with the connected devices.

The tiles 4102A, 4102B, 4102C, 4102D, 4102E include various interactivecontrol elements or objects. The interactive control elements or objectscan be selected by a user by providing input using a touchscreen, usingan input device, or any other appropriate input device. For example, aninput device (a touchscreen, a mouse, or other input device) may receiveinput corresponding to a user's selection of an interactive controlelement. The user may cause the input to occur by selecting theinteractive control element using the input device, such as by tapping aportion of a touchscreen corresponding to the interactive controlelement, selecting the interactive control element using a mouse orother pointing device, or the like.

The tiles 4102A, 4102B, 4102C, 4102D, 4102E each include respectiveicons 4104A, 4104B, 4104C, 4104D, 4104E. As described above with respectto FIG. 40, the icons 4104A, 4104B, 4104C, 4104D, 4104E can be customfor a specific device, default for the type of device, or genericdepending on the meta-information provided by each respective device.For example, the icon 4104A includes a specific, custom icon relating tothe specific device portrayed in tile 4102A, which includes a homeautomation switch. In another example, the icon 4104B includes a defaulticon for a smart television.

The tiles 4102A, 4102B, 4102C, 4102D, 4102E further include names 4106A,4106B, 4106C, 4106D, 4106E identifying the different network devices. Insome embodiments, the names may be automatically created by the gatewaydevice interface. In some embodiments, a user may create the names ormay edit the automatically created names.

The tiles 4102A, 4102C, and 4102D also include interactive controlelements or objects 4108A, 4108C, and 4108D. For example, tile 4102Acorresponds to a home automation switch and shows an interactive controlelement 4108A depicted as a power button. The interactive controlelement 4108A is selectable by a user to turn the automation switch onand off. Similarly, tile 4102C shows interactive control element 4108Cdepicted as a power button 4108C for turning a lamp light on and off. Insome embodiments, the device specific interface object (e.g., icon4104A) includes a control object, and is selectable to control a deviceusing one or more primary control capabilities. For example, in someembodiments, the icon 4104A may be selectable to turn the switch on oroff. The interactive control elements 4108A, 4108C, and 4108D provide auser with limited control of the respective devices. A user is requiredto access a proprietary interface for the different devices in the eventthe user desires more control functionality. For example, the user mayselect a device interface interactive element 4124A, 4124B, 4124C,4124D, or 4124E to access a proprietary interface that allows fullcontrol of the respective devices. Upon receiving input corresponding toa selection of a device interface interactive element 4124A, 4124B,4124C, 4124D, or 4124E, the gateway device interface may cause theselected proprietary interface to launch. The access device running thegateway device interface may then cause a display to toggle from thegateway device interface to the selected proprietary interfacecorresponding to the respective device. For example, the access devicemay launch a proprietary application that runs the selected proprietaryinterface (e.g., a smart light bulb proprietary application running aproprietary interface for controlling or accessing only certain lightbulbs, an internet of things security system proprietary application, orthe like). Examples of proprietary interfaces are described with respectto FIGS. 42-44.

In some embodiments, one or more of the devices connected to the gatewaydevice may not have provided control capabilities to the gateway device.For example, the gateway device may not have obtained controlcapabilities for the smart television (TV) depicted in tile 4102B andthe smart phone depicted in tile 4102E. In such cases, an interactivecontrol element may not be displayed in the tiles 4102B and 4102E. Insome embodiments, usage statistics 4108B and 4108E may be displayed inthe tiles 4102B and 4102E. As described above, the usage statistics mayinclude various behavioral characteristics of the smart TV and smartphone, such as bandwidth usage, network usage time, activity type, orany other information. In some embodiments, usage statistics may bedisplayed for any device depicted in the gateway device interface byselecting respective interactive elements 4110A, 4110B, 4110C, 4110D, or4110E. The user may be able to monitor activity of the devices based onthe usage statistics. For example, the user may include a parent in ahousehold that includes the local area network. The parent may haveadministrative privileges to control settings of the gateway deviceinterface (e.g., by using specific login credentials). The parent maymonitor usage of a child's smart phone to determine whether the child isusing the phone more than an allotted amount of time.

In some embodiments, a user may remove a device from the local areanetwork by selecting interactive elements 4112A, 4112B, 4112C, 4112D,4112E. In one example, selection of interactive element 4112E maydisconnect the smart phone from the gateway device. For example, theparent may decide that the child has used the smart phone for more thanan allotted amount of time, and may select the interactive element 4112Eto prevent the smart phone from connecting to the local area network.

In embodiments, one or more of the tiles 4102A, 4102B, 4102C, 4102D,4102E may display a current state or status of network devices. Forexample, a status in tile 4102B may indicate a television-specificstatus, such as “Television Was Last Viewed at 10 AM.” In additional oralternative embodiments, tiles 4002 can convey status information abouta network device, including, but not limited to, a firmware version, alast known firmware update status, connectivity to cloud status,registration status (e.g., an indication that the network device has akey or does not), a primary mode of the network device (e.g., on, off),a secondary mode of the device (e.g., standby, high, low, eco), aschedule, and settings for the device. In another example, a status inthe tile 4102C for the bedroom lamp device can indicate a brightnesslevel for the lamp. For example, the status may indicate that the lampis at 0%, corresponding to the lamp being turned off. In someembodiments, a status may be reflected by the interactive controlelements 4102A, 4102C, or 4102D. For example, the interactive controlelement 4108D displayed as a power button for the space heater is shadedto indicate the space heater is currently in an on state.

The graphical interface display 4100 further includes selectable iconsand links 4114, 4116, 418, 4120, and 4122 outside of the tile displayarea. For example, the refresh icon 4114 can be selected to refreshinformation presented in display 4100, such as, for example, status andstate information displayed in tiles 4102A, 4102B, 4102C, 4102D, 4102E.The status information for a device can be refreshed automatically on aperiodic basis, and may also be manually updated using selectablerefresh icon 4114. In some embodiments, selection of the refresh icon4114 may cause any newly discovered devices to be displayed in thedisplay 4100. The edit link 4116 can be selected to edit the list oftiles 4102A, 4102B, 4102C, 4102D, 4102E. For example, edit link 4116 canbe selected to sort or re-order the sequence of tiles 4102A, 4102B,4102C, 4102D, 4102E displayed in display 4100. Edit link 4116 can alsobe selected to delete one of the tiles 4102A, 4102B, 4102C, 4102D, 4102Ein cases where a user no longer wants to view a given tile. Devices icon4118 can be selected to list other discovered devices connected to thegateway device that are not currently displayed in the graphicalinterface display 4100.

The gateway settings icon 4120 may be selected to access the gatewaydevice settings. For example, a user may provide input to the gatewaydevice interface to change one or more settings of the gateway device.In one example, a user may select an option to change a communicationfrequency band upon which to exchange communications with devices, suchas from 2.4 GHz to 5 GHz, from 5 GHz to 2.4 GHz, or to and from anyother appropriate frequency band. In another example, a user may selectoptions to adjust the access credentials used to access the gatewaydevice (e.g., SSID name of network and password), to adjust credentialsused to access the gateway device interface or the settings of thegateway device (e.g., login credentials for the gateway deviceinterface), or the like. Other settings may be adjustable by a user,such as remote access settings, time settings, parental controls, backupoptions, or any other settings for the gateway device. The more icon4024 can be selected to access additional features of the gateway deviceinterface.

As previously described, the interactive control elements 4108A, 4108C,and 4108D of the gateway device interface provide a user with limitedcontrol of the respective devices. In the event a user wants to controla device using a control capability that is outside of the one of theprimary control capabilities, the user can be directed to a proprietaryinterface for the device. The user may select a device interfaceinteractive element 4124A, 4124B, 4124C, 4124D, or 4124E to access aproprietary interface that allows full control of the respectivedevices. Upon receiving input corresponding to a selection of a deviceinterface interactive element 4124A, 4124B, 4124C, 4124D, or 4124E, thegateway device interface may cause the selected proprietary interface tolaunch. The access device running the gateway device interface may thencause a display to toggle from the gateway device interface to theselected proprietary interface corresponding to the respective device.In some embodiments, the gateway graphical interface may include thefull set of control capabilities for one or more of the devicesconnected to the gateway.

FIGS. 42-44 illustrate different examples of graphical interfacedisplays 4200, 4300, 4400 of a proprietary interface that can beimplemented by an access device (e.g., through execution of a mobileapplication, a web-based application, or any other appropriate program)to allow a user to control and/or configure devices that utilize theproprietary interface. A proprietary interface may be specific to asingle device, or may be specific to multiple devices of a similar orsame type (e.g., built by a same manufacturer). For example, networkdevices 3902, 3904, 206 may be made by a manufacturer that provides theproprietary interface for the network devices 3902, 3904, 3906. In oneexample, a company may make a line of Internet of Things (IoT) devicesthat provide home automation capabilities. The company may provide aproprietary application (e.g., mobile application, web-basedapplication, or the like) that generates a proprietary graphicalinterface that can be used to control and monitor the IoT devices. Theproprietary graphical interface allows full control of the differentdevices that are controllable using the proprietary graphical interface.

Throughout FIGS. 42-44, displays are shown with various tiles,interactive elements, icons, links, command regions, windows, toolbars,submenus, drawers, and buttons that are used to initiate action, invokeroutines, monitor network devices, configure network devices, controlnetwork devices, or invoke other functionality. The initiated actionsinclude, but are not limited to, displaying a current state of a networkdevice, displaying historical data associated with a network device,displaying a last known status of a network device, selecting one ormore primary settings for a network device, selecting one or moresecondary or tertiary settings of a network device, and other inputs andgestures. For brevity, only the differences occurring within thefigures, as compared to previous or subsequent ones of the figures, aredescribed below.

FIG. 42 illustrates a graphical interface display 4200 of a proprietaryinterface of an application executing on an access device. For example,the proprietary interface may be part of an application provided by amanufacturer of the hall switch shown in tile 4202A, the motion sensorshown in tile 4202B, and the lamp shown in tile 4202C. The hall switchand the lamp are also shown in the graphical interface display 4100 ofthe gateway device interface. The motion sensor is not shown in thegraphical interface display 4100 of the gateway device interface. Forexample, the motion sensor may not be connected to the gateway, and thusmay not be displayed by the gateway device interface. The motion sensormay be connected to the access device using one or more communicationsignals that are independent of the signals provided by the gatewaydevice. In one illustrative example, the motion sensor may use Bluetoothsignals to connect to the access device instead of WiFi signals providedby the gateway device. The graphical interface display 4200 may bedisplayed as a result of a user selecting one of the device interfaceinteractive elements 4124A or 4124C.

The graphical interface display 4200 includes tiles 4202A, 4202B, and4202C for interacting with network devices in a network using controlcapabilities of the network devices. The interactions can includemonitoring and controlling the network devices. The tiles 4202A, 4202B,and 4202C each include respective icons 4204A, 4204B, and 4204C, andrespective names 4206A, 4206B, and 4206C identifying different networkdevices. For example, tile 4202A corresponds to a switch and shows aninteractive control element depicted as a power button 4208A for turningthe switch on and off. Similarly, tile 4202C shows a power button 4208Cfor turning a lamp on and off.

Display 4200 is a visual interface usable to monitor and control one ormore network devices using a full set of control capabilities. Display4200 includes tiles for discovered network devices. The tiles mayinclude an icon representing a network device and a name for the networkdevice. In the example of FIG. 42, display 4200 includes tiles 4202A,4202B, and 4202C corresponding to three different network devices. Inparticular, tile 4202A is configured to control a switch network device.Tile 4202A includes an icon 4204A representing the switch and a name4206A for the switch. In embodiments, tiles 4202A, 4202B, and 4202C arecustomizable by developers and/or manufacturers of the network devices.For, example, name 4206A is customized to indicate the name of aspecific type of switch (e.g., ‘Hall switch’). Tile 4202A also includesan interactive element 4208A that can be selected to control the switch.For example, interactive element 4208A can be selected with a click orpress to turn the switch on and off. Tile 4202B includes an icon 4204Bfor a sensor network device. Tile 4202B includes an icon 4204Brepresenting the sensor and a name 4206B for the sensor (e.g., ‘MyMotion Sensor’). In some embodiments, tiles 4202 are customizable bydevelopers and/or manufacturers of the network devices to display acurrent state or status of network devices. For, example, tile 4202B iscustomized to indicate a sensor-specific status (e.g., ‘Motion sensed at11:05 AM’). In additional or alternative embodiments, tiles 4202 canconvey status information about a network device, including, but notlimited to, a firmware version, a last known firmware update status,connectivity to cloud status, registration status (i.e., an indicationthat the network device has a key or does not), a primary mode of thenetwork device (i.e., on, off), a secondary mode of the device (i.e.,standby, high, low, eco), a schedule, and settings for the device (see,e.g., schedule 4434 in FIG. 44).

Depending on the type and capabilities of a network device, the networkdevice's tile can display a status or state of the device. For example,FIG. 42 shows how status 4210B indicates when a motion sensor lastdetected motion. In an embodiment, the display 4200 is flexible and candynamically display the most appropriate tile 4202 for the environmentin which the application is run. For instance, the display 4200 enablesa mechanism wherein a user can pre-configure a tile 4202B to display themost appropriate message for a connected sensor. For example, a user canconnect any one of a plurality of types of sensors (e.g., water,humidity, temperature/heat, air pressure, light, sound, smoke, carbonmonoxide, etc.) to a network. As shown in FIG. 42, instead of presentinga default interface stating a generic message such as ‘the sensor hasdetected something at a certain time,’ the tile 4202B can be configuredto provide a sensor-specific status (e.g., ‘Motion sensed in the livingroom 11:05 AM’ or ‘someone is in the living room.’). Similarly, tilesfor other types of sensor devices can be configured to presentsensor-specific states or status messages such as, for example, ‘theleak detector at a certain location has detected elevated levels ofmoisture’ or ‘the smoke detector has detected smoke in the kitchen.’

As shown in FIG. 42, the status 4210C in the tile 4202C for the lampdevice can indicate a brightness level for the lamp. In the exemplarydisplay 4200, status 4210C indicates that the lamp is at 0%, whichcorresponds to the lamp being turned off. In an embodiment, this statusis also reflected by the power button 4208C for the lamp not being litup or shaded/bolded. When a network device has extended capabilities,such as secondary or tertiary functionalities, an interactive element4212 can be selected to expand and contract a menu includingcontrollable settings for the capabilities. The menu can be, forexample, a full drop down menu or drawer with interactive elements forsetting the extended capabilities of the network device. An example ofsuch a full menu and drawer is provided in FIG. 44, which is discussedbelow. The full menu and drawer are examples of control capabilitiesthat may not be provided in the gateway graphical interface in somecases.

The display 4200 further includes selectable icons and links 4214, 4216,4218, 4220, 4222, and 4224 that are located outside of the tile displayarea. In the example of FIG. 42, refresh icon 4214 can be selected torefresh information presented in display 4200, such as, for example,status and state information displayed in tiles 4202A, 4202B, and 4202C.For instance, the status 4210B in tile 4202B for the motion sensor canbe refreshed on an automatic, periodic basis, in addition to beingmanually updated when refresh icon 4214 is selected. Similarly, thebrightness status 4210C in tile 4202C for the lamp can be updated whenrefresh icon 4214 is selected. The edit link 4216 can be selected toedit the list of tiles 4202A, 4202B, and 4202C. For example, edit link4216 can be selected to sort or re-order the sequence of tiles 4202A,4202B, and 4202C displayed in display 4200. Edit link 4216 can also beselected to delete one of the tiles 4202A, 4202B, and 4202C in caseswhere a user no longer wants to view a given tile. Devices icon 4218 canbe selected to list discovered network devices in a network.

Rules icon 4220 can be selected to display rules pertaining to networkdevices. For example, rules icon 4220 can be selected to display a rulethat turns on the lamp of tile 4202C for a specified duration when themotion sensor of tile 4202B detects motion. For example, a user canselect rules icon 4220 and create or edit a rule that turns on a porchlight for a certain number of minutes when a motion sensor detectsmotion near the porch. In this way, rules can relate functionalities ofmultiple network devices to each other. News icon 4222 can be selectedto review news items, such as news associated with network devicesand/or the application. For instance, news icon 4222 can be selected toview announcements and news items relevant to network devices controlledusing tiles 4202A, 4202B, and 4202C, and/or information relevant to theapplication, such as notifications of available tile updates. The moreicon 4224 can be selected to access additional features of theapplication.

FIG. 43 shows another example of a graphical interface display 4300 ofan application executing on an access device. The graphical interfacedisplay 4300 displays a space heater, which is also displayed in thegateway device interface 4022 provided in the graphical interfacedisplay 4100. In some examples, the application and graphical interfacedisplaying the graphical display 4300 may be launched in response to auser selecting the device interface icon 4124D in order to have accessto more control capabilities of the space heater. The graphicalinterface display 4300 shows the status of the space heater within tile4302A. For example, tile 4302A can show that the heater is turned bycoloring, animating, or shading a portion of power button 4308A. In oneembodiment, a portion of power button 4308A can be shaded green orbolded to indicate that the space heater has been turned on. In someembodiments, a submenu 4314 can be exposed when the power button 4308Ais selected. The submenu 4314 provides one or more selectable objectscorresponding to secondary settings of the heater. The secondarysettings of the submenu 4314 provide further functionality that isrelated to the primary function of turning the network device on. In theexample of FIG. 43, submenu 4314 is a drawer that shows secondarysettings corresponding to high, low, and eco operational modes for thespace heater when the power button 4308A is selected to turn the spaceheater on. In some cases, the submenu 4314 is displayed momentarily whenthe power button 4308A is selected, and may close after a certain periodof time. For example, submenu 4314 may be temporarily displayed withinthe context of the heater being turned on. The submenu 4314 may includea default choice for a secondary function space heater (e.g., economicor eco mode), and can also be used to select other secondary functions(e.g., high or low). In this way, with a single input of selecting orpressing power button 4302A, both primary and secondary functionalitiescan be set using a single input. Additional functionalities andcapabilities of the space heater can be displayed by selectinginteractive element 4312 to expand a full menu of control capabilitiesof the space heater. An example of the full menu is shown in FIG. 44.

FIG. 44 shows another example of a graphical interface display 4400 ofan application executing on an access device. In the example shown inFIG. 44, the status indicator 4310 indicates that the space heater is instandby mode. This status can be visually reinforced by the power button4308A being lightly shaded. In some examples, the status indicator 4310and power button 4308A may transition when the status of the spaceheater is changed. For example, when the power button 4308A is selectedto turn on the space heater, and secondary mode of ‘low’ is selected insubmenu 4314, the power button 4308A may become illuminated (e.g.,shaded, change color, or any other display feature). Additionally, thestatus indicator 4310 may transition from ‘standby’ to ‘low’ to reflectthe selection in submenu 4314.

The graphical interface display 4400 also displays detailed status andextended functionalities of the space heater within a full menu 4426accessed using tile 4302A. For example, the full menu 4426 can bedisplayed when interactive element 4312 is selected. In the example ofFIG. 44, the full menu 4426 can include interactive elements 4430, 4432,4436 for selecting settings or actions corresponding to tertiary orextended functions of the space heater. In particular, the exemplaryfull menu 4426 is shown displaying temperature readings 4428 (e.g., roomtemperature and external temperature), thermostat setting 4430, an autooff setting 4432, schedule information 4434 for the space heater, and afrost protection setting 4436. In the example of FIG. 44, the auto offsetting 4432 is a selectable duration (e.g., a number of minutes, anumber of hours, a number of hours and minutes, or any other appropriatetime frame). In some embodiments, the auto off setting 4432 can bechanged using a scroll wheel interface element, directly entered astext, or changed using another input mechanism. Full menu 4426 can beimplemented as a drawer showing historical data associated with anetwork device, such as, for example, past temperature readings andpatterns of temperature readings. The historical data may be used todetermine tertiary settings or functionalities, such as, for example,the frost protection setting 4436. As shown, the frost protectionsetting 4436 can be toggled on an off within the full menu 4426. Asillustrated in the graphical display 4400, the space heater willautomatically turn on if the room temperature falls below 40 degreesFahrenheit. In some cases, certain extended capabilities can be tied torules. For example, the rule for frost protection can be changed byselecting rules icon 4020 in order to edit the logic for the rule.

As shown in the embodiments of FIGS. 41-44, the status or state of anetwork device can be indicated within the tile with text and/orgraphically. For instance, coloring or shading of an interactiveelement, such as power button 4308A, can indicate whether a networkdevice is connected to the network, powered off or on, restarting, or ina standby, sleep, or hibernation mode. In one example, part or all ofthe power button 4308A in the tile 4302A can be shaded green when thespace heater is powered on, can be shaded red or un-shaded when thespace heater is off, and can be shaded yellow or lightly shaded when thespace heater is in standby mode. In additional or alternativeembodiments, a power button can pulsate or slowly blink when the deviceis in a transitional state. Examples transitional states include bootingup, restarting, connecting to a network, re-connecting to the network,or the like.

FIG. 45 illustrates an embodiment of a process 4500 of discoveringdevices connected to a gateway device and determining controlcapabilities and meta-information of the connected devices. In someaspects, the process 4500 may be performed by a computing device, suchas a gateway device 110, 112, or 3910 shown in FIGS. 1, 39 and 40. Whilespecific examples may be given of a gateway device, such as a router,performing the process 4500, one of ordinary skill in the art willappreciate that other devices may perform the process 4500.

Process 4500 is illustrated as a logical flow diagram, the operation ofwhich represent a sequence of operations that can be implemented inhardware, computer instructions, or a combination thereof. In thecontext of computer instructions, the operations representcomputer-executable instructions stored on one or more computer-readablestorage media that, when executed by one or more processors, perform therecited operations. Generally, computer-executable instructions includeroutines, programs, objects, components, data structures, and the likethat perform particular functions or implement particular data types.The order in which the operations are described is not intended to beconstrued as a limitation, and any number of the described operationscan be combined in any order and/or in parallel to implement theprocesses.

Additionally, the process 4500 may be performed under the control of oneor more computer systems configured with executable instructions and maybe implemented as code (e.g., executable instructions, one or morecomputer programs, or one or more applications) executing collectivelyon one or more processors, by hardware, or combinations thereof. Asnoted above, the code may be stored on a machine-readable storagemedium, for example, in the form of a computer program comprising aplurality of instructions executable by one or more processors. Themachine-readable storage medium may be non-transitory.

At 4502, the process 4500 includes detecting, by a gateway device, anetwork device connected to the gateway device, wherein the gatewaydevice provides communication routing for the network device. Forexample, the gateway device may include the gateway 3910 and the networkdevice may include the network device 3902. The process may includedetecting, by the gateway device, multiple network devices connected tothe gateway. In some examples, the gateway device may detect connectednetwork devices based on an identifier (e.g., a MAC address, a serialnumber, or other suitable unique identifier) associated with theconnected network devices. For example, the gateway device may be awarethat a network device with identifier A is connected to the gatewaydevice, such as using a device directory (e.g., device directory 4002).In some examples, the gateway device may detect the presence of networkdevices that are connected to the gateway device, or network devicesthat are not yet connected to the gateway device. For example, thegateway device may detect that the network device is withincommunication signal range of the gateway device.

At 4504, the process 4500 includes probing the network device formeta-information and control capabilities of the network device. Forexample, upon detecting the network device, the gateway device may senda communication to the network device. The communication may include arequest to connect to the network device in the event the network deviceis not already connected to the gateway device. The communication mayfurther include a query requesting meta-information and controlcapabilities of the network device. In some embodiments, a request toconnect and a query for meta-information and/or control capabilities maybe sent in separate communications. For example, the query may be sentafter the network device is connected to the gateway device.

At 4506, the process 4500 includes receiving the meta-information andthe control capabilities of the network device, wherein themeta-information includes identifying information of the network device.In some embodiments, the gateway device may populate or update a devicedirectory (e.g., device directory 4002) record for the network devicewith the received meta-information and the control capabilities, andalso with further information determined using the meta-information andthe control capabilities. In some examples, the meta-informationincludes identifying information of the network device that is exposedor published for access by the gateway device, as described above withrespect to FIG. 40.

At 4508, the process 4500 includes determining a device specificinterface object for the network device using the meta-information. Insome examples, a device specific interface object may include an iconthat identifies the specific device in a gateway device interface. Forexample, an access device, or other device, may execute a devicespecific interface object module to display the device specificinterface object that is specific to the network device. Depending onthe detail of the meta-information, the device specific interface objectmay be a generic icon, a default icon, or an icon that is custom andspecific to the network device.

The gateway device can determine the device specific interface objectfor the network device depending on the information included in themeta-information exposed by the network device. In some embodiments, themeta-information includes a device specific interface object module. Forexample, the network device may store its own device specific interfaceobject module, and may send it to the gateway device with themeta-information. The device specific interface object module mayinclude instructions or code that can be executed by a processor tocause the device specific interface object to be determined and/ordisplayed. The gateway device can use the received device specificinterface object module to determine the device specific interfaceobject for the network device.

In some embodiments, the meta-information includes a unique identifierfor the network device, and the process 4500 may further includetransmitting the unique identifier to a cloud server, receivinginformation identifying a type of the network device, and determiningthe device specific interface object for the network device using theinformation identifying the type of the network device. For example, asdescribed above with respect to FIG. 40, the gateway 3910 may sendidentifying information to a server in the cloud network 3914. The cloudnetwork 3914 may be maintained by a manufacturer of the gateway 3910, amanufacturer of the network device 3902, a web server, or other externalnetwork provider.

In some embodiments, determining the device specific interface objectfor the network device further includes monitoring behaviorcharacteristics of the network device, determining a type of the networkdevice based on the monitored behavior characteristics, and determiningthe device specific interface object for the network device using thedetermined type of the network device.

At 4510, the process 4500 includes determining one or more primarycontrol capabilities corresponding to the network device. The gatewaydevice can determine the one or more primary control capabilitiescorresponding to the network device based on the received controlcapabilities. In one example, the gateway device may determine that theprimary control capabilities of the network device include turning thenetwork device on and off.

At 4512, the process 4500 includes transmitting the device specificinterface object module and the one or more primary controlcapabilities, wherein the device specific interface object module andthe one or more primary control capabilities facilitate display of agateway device interface. The gateway device interface displays thedevice specific interface object and a control object. The controlobject allows control of the network device using the one or moreprimary control capabilities. The one or more primary controlcapabilities include less than all control capabilities of the networkdevice that are provided in a proprietary interface of the networkdevice, as illustrated in FIGS. 41-44.

In some embodiments, the process 4500 further includes transmitting themeta-information of the network device to a second network device. Thesecond network device can utilize the meta-information, for example, todisplay information related to the network device in graphicalinterfaces associated with the devices. For example, a network printermay receive meta-information related to a network-connected camera. Agraphical interface used to display printer features and functionalitymay use the meta-information to display information related to thenetwork-connected camera.

In some embodiments, the device specific interface object includes thecontrol object, and the device specific interface object is selectableto control the network device using the one or more primary controlcapabilities. For example, a displayed icon for the network device mayalso be selectable to control the network device. In some embodiments,the gateway device interface includes a proprietary interface object.The proprietary interface object is selectable to launch the proprietaryinterface of the network device to allow execution of a controlcapability outside of the primary control capabilities. In someembodiments, the gateway device interface displays informationcorresponding to a plurality of network devices connected to the gatewaydevice.

In some embodiments, a modular visual interface framework may beutilized to dynamically and implicitly provide visual interface modulesto an access device 108 so that the access device 108 can be used tocontrol network devices within a network without having to install a newapplication or a version of an application for each network device. Thevisual interface modules can enable a user of the access device 108 toremotely control network devices within a network without having tophysically interface with the network device. In certain embodiments, anapplication installed on the access device 108 can have a graphicalinterface, and the application can be configured to execute one or morevisual interface modules usable to control respective network devices ina local area network. The visual interface modules, when executed by anapplication, can render a visual interface in the graphical interface toenable control of operation of the network device. In some embodiments,the visual interface module can be specific to a given network device.

The visual interface rendered for a visual interface module can be amodular tile that includes information identifying a respective networkdevice and includes interactive areas or interactive elements forcontrolling and/or monitoring the network device on a network. Thevisual interface can provide information about a status of the networkdevice corresponding to the tile. The status of a network device may beany changeable variable of that particular network device. For example,the status of a network device may include a state of the network deviceitself (e.g., on or off) or how the network device is situated withinthe network with respect to the other network and other devicesthroughout the network. In certain embodiments, the status can include avalue, a state, or other unit of measure corresponding to a setting oran attribute related to operation of a device. The setting or theattribute can be adjustable within a range of values or betweendifferent states. For example, the device can be a light bulb and thestatus can include a value corresponding to brightness (e.g., apercentage of total brightness) emitted by the light bulb when the lightbulb is powered-on.

The visual interface can include one or more interactive elements orinteractive areas to control one or more settings and/or attributesrelated to operation of the network device corresponding to the visualinterface. The settings and/or attributes can correspond tofunctionalities or features of the network device. The functionalitiescan include, for example, powering the network device on and off, oradjusting a setting or an attribute of the network device. In someembodiments, the visual interface may enable a user to initiateadditional actions by interacting with an interactive area for athreshold time period. The visual interface can be updated to reflectthe status of the network device with respect to the adjustment of oneor more attributes and/or settings. Operation and implementation of themodular visual interface framework is described below with reference toFIG. 46. In particular, FIGS. 48-53 show examples of visual interfacesthat enable a user to control operation of network devices and todetermine a status of the network devices. The visual interfaces shownin the FIGS. 48-53 may enable a user to determine a status of networkdevices and may enable the user to manage those network devices based ontheir statuses.

FIG. 46 illustrates an embodiment of a process 4600 for providing avisual interface module for controlling a network device. As shown, theprocess 4600 may be performed by one or more computing devices, such asthe network device 102, a server associated with cloud network 114, orthe access device 108 described above with reference to FIG. 1. In someembodiments, the network device 102 is associated with a home automationnetwork, such as the local area network 100 described above with respectto FIG. 1. Process 4600 is illustrated as a data flow diagram, theoperation of which represents operations that can be implemented inhardware, computer instructions, or a combination thereof. Gateway 110is connected to cloud network 114, and allows network device 102 toconnect to the cloud network 114, the Internet, or other externalnetworks via gateway 110. In some embodiments, the network device 102may be a home automation network device that allows a user to access,monitor, control, and/or configure various appliances, such as atelevision, radio, light bulb, microwave, iron, fan, space heater,sensor, and/or the like. In some embodiments, the user can monitor andcontrol network devices by interacting with a visual interface renderedby the gateway 110 (i.e., a web page for gateway 110), a visualinterface rendered on display 4622 of the access device 108, or a visualinterface rendered by the network device 102.

In an embodiment, an application may be run on the access device 108.The application may cause the access device 108 to present a display4622 with a modular visual interface for each network device accessibleon the local area network 100. When the application is run on the accessdevice 108, the access device 108 can access a cache 4602.

The cache 4602 can be a local cache located in onboard storage of theaccess device 108. The cache 4602 can contain a known interface list4620 with records 4624, 4626 and 4628 including interface informationfor different, known types of network devices. As shown, each of records4624, 4626 and 4628 can include a device type, a unique interface moduleID, and controls information. The known interface list 4620 can includea record for each device known by the access device 108 to exist on thelocal area network 100. When the application is run on the access device108, the access device 108 can access the known interfaces 4620 in thecache 4602 to present the display 4622, which lists modular interfacesfor each network device on the local area network 100. In an example,the display 4622 can include a modular tile for each connected networkdevice having an interface in the known interface list 4620. Examples ofcommunications used to populate cache 4602 are described in thefollowing paragraphs.

The process 4600 can include utilizing communication 4606 to register avisual interface module for a network device 102 with a server of cloudnetwork 114. For simplicity, communication 4606 is shown as a directcommunication between network device 102 and cloud network 114. However,it is to be understood that in an embodiment, communication 4606 can besent from a manufacturer of network device 102 to cloud network 114. Inan additional or alternative embodiment, communication 4606 is sent fromthird party interface developer to cloud network 114. For example, athird party developer of a visual interface module for network device102 may initiate communication 4606 to cloud network 114. In the exampleof FIG. 46, communication 4606 includes registration information for thenetwork device 102. For example, communication 4606 can include a uniquedevice ID for network device 102. In some embodiments, the registrationinformation may identify one or more capabilities of network device 102.The registration information can include a unique identifier for thenetwork device, a default name of the network device, one or morecapabilities of the network device, and one or more discovery mechanismsfor the network device. In one example, communication 4606 can include aresource bundle corresponding to network device 102. The resource bundlecan be embodied as a structured folder structure whose contents defineall visual and interactive elements/areas in a tile. For example, aresource bundle can be a zip file sent from a device manufacturer or athird party developer that is submitted or uploaded to cloud network114. The resource bundle includes a unique device ID and files defininggraphical content of a visual interface module. The graphical contentcan include definitions of interactive elements/areas for the interfacemodule. The resource bundle can include templates defining interactivecontrol states for each of the interactive elements, languagetranslations for tile text, any menus for the tile, and graphicalcontent of the menus. For example, the resource bundle can definetemplates, text, and graphical content using a markup language, such asHTML5.

At 4606, the process 4600 includes transmitting an indication thatnetwork device 102 is associated with the network. For example, networkdevice 102 may transmit the indication to the server of the cloudnetwork 114. In some embodiments, transmitting may include transmittinga unique identifier (ID) for the network device 102. For example, thenetwork device 102 may send a communication to the server indicating aunique interface module ID for the network device 102. In suchembodiments, the server may then determine that a match between theunique interface module ID and a known interface exists. The cloudnetwork 114 can include a data store 204 of known interfaces. The accessdevice 108 can download a visual interface module identified in datastore 204 from the cloud network 114, which can be used to render amodular interface within display 4622. In an embodiment, data store 4604can be a tile database where each record in the database is uniquelyidentified by a tile ID.

Cloud network 114 can use the unique device ID to determine an interfacemodule for network device 102. As shown in FIG. 46, cloud network 114can access a data store 4604 of visual interface modules. A plurality ofuniquely identified interface modules can be stored in data store 4604.For example, each interface module in data store 4604 can be associatedwith a unique interface module ID. In an embodiment, data store 4604 isa database configured to store modular tiles for a plurality of networkdevices, with each of the stored modular tiles being identified by aunique tile ID. For instance, the network device 102 having a uniquedevice identifier may be matched with an existing interface module basedon comparing information received from the network device 102 withinformation stored in data store 4604. In cases where an existinginterface module for network device 102 is not found in data store 4604,cloud network 114 can use information in a resource bundle for thenetwork device 102 to generate an interface module, where the resourcebundle is provided as part of a registration process for a given networkdevice. The generated interface module can then be stored in data store4604 and assigned a unique interface module ID. In some embodiments,information in the resource bundle can be used to update an existinginterface module stored in data store 4604. After determining theinterface module for network device 102, cloud network 114 sendscommunication 4608 to network device 102 in order to provide a uniqueinterface module ID to the network device 102. In one embodiment,communication 4608 can include a unique tile ID corresponding to amodular tile for network device 102 that is stored in data store 4604.In some embodiments, communication 4608 includes a unique tile IDcorresponding to a modular tile defined for network device 102. Uponreceiving communication 4608 with the unique interface module ID (i.e.,a unique tile ID), the network device 102 can store the unique interfacemodule ID. In one embodiment, for example, the unique interface moduleID can be stored by an interface device 4601 of the network device 102that is configured to provide the interface module ID to an accessdevice or gateway. In an embodiment, the interface device 4601 isimplemented as a ‘smart module’ in hardware and firmware, such as, forexample, a system on a chip (SOC) integrated into the network device102.

The interface device 4601 can include flash memory and dynamic randomaccess memory (DRAM). The flash memory may be used to store instructionsor code relating to an operating system, one or more applications, andany firmware. The flash memory may include nonvolatile memory so thatany firmware or other program can be can updated. In the event theinterface device 4601 loses power, information stored in the flashmemory may be retained. The DRAM of the interface device 4601 may storevarious other types of information needed to run the interface device4601, such as all runtime instructions or code. The flash memory or DRAMor a combination thereof may include all instructions necessary tocommunicate with network device 102.

The process 4600 can include sending, from the access device 108,intra-network communication 4610 including a query, to the networkdevice 102. The query can be a request for information such as a queryfor capabilities, a request for an identity of the network device 102,and/or a request for a unique interface module ID. For example,communication 4610 can be sent from access device 108 to network device102 to query network device 102 about its identity. In response to thequery sent from access device 108, the process 4600 can includereceiving intra-network communication 4612 at the access device 108 withdevice information for the network device 102. According to anembodiment, in response to the query, the network device 102 can sendcommunication 4612 to inform the access device 108 of the identityand/or capabilities of the network device 102. For instance, in responseto receiving the query, the network device 102 may send communications4612 to the access device 108 with at least a unique interface moduleID. The process 4600 can include utilizing intra-network devicecommunications 4610 and 4612 as part of a discovery process for thenetwork device 102. For example, when the network device 102 isinitially connected to the network, it and access device 108 canautomatically exchange communications 4610 and 4612 to provide theaccess device 108 with information that can be used to determine abasic, default visual interface stored in cache 4602.

Within the context of a modular tile framework, embodiments candynamically render a functional user interface without having todownload the appropriate interface template from a remote server, suchas a server associated with the cloud network 114, in order to control anewly discovered network device. These embodiments can be used in caseswhere a connection to the Internet or the cloud network 114 isunavailable or unreliable, and immediate use of a newly discoverednetwork device is desired. In this case, an application on the accessdevice 108 or a stationary device such as gateway 110 could, based oncertain information received from the network device 102, dynamicallyrender a functional interface for immediate use. Such a functionalinterface may not be the ideal, visually optimized, interface that isdownloadable from the cloud network 114. However, such a functionalinterface will suffice until the application is able connect to theInternet and/or the cloud network 114 and subsequently download theappropriate and visually optimized interface module for the networkdevice 102.

In some embodiments, communication 4612 may be received when the networkdevice 102 is rebooted (e.g., powered on, reset or restored to defaultsettings, or the like). For example, when the network device 102 isrebooted, it may broadcast one or more messages on the local areanetwork to discover whether there are any access devices in the localarea network 100. For example, communication 4612 may be broadcastaccording to a UPnP protocol during a discovery process. The networkdevice 102 may receive communications 4610 from access device 108indicating that it is located within the local area network andinterrogating network device 102 about its functionalities. That is,after receiving a broadcast message from network device 102, accessdevice 108 may then query network device 102 by sending communication4610 in order to receive the communication 4612 including informationabout the network device.

After receiving communication 4612, if the access device 108 can accessthe cloud network 114, it sends a communication 4614 to the cloudnetwork 114 as a request for an interface module for the network device102. Communication 4614 is described in further detail below. Otherwise,if the access device 108 cannot access the cloud network 114, the accessdevice 108 looks up the unique interface module ID received from thenetwork device 102 in cache 4602. As discussed above, cache 4602 can bea local cache stored on the access device 108. Basic properties forknown interfaces can be stored in cache 4602 as a device type andcontrols information. These basic properties can include, for example, adefault icon, a default name, and interactive elements or interactiveareas for controlling one or more primary functionalities of a networkdevice. The primary functionalities can include, for example, poweringthe network device on and off. The basic properties can also includecontrols information for secondary functionalities.

In some embodiments, when the access device is connected to the cloudnetwork 114, the access device 108 sends communication 4614 to query thecloud network 114 about network device 102. The communication 4614 caninclude at least the unique interface module ID for the network device102. At this point, the cloud network 114 can compare the uniqueinterface module ID of the network device 102 to known interface moduleIDs stored in data store 4604 in order to determine that there is amatch between the unique interface module ID sent with communication4614 and a known interface module. If the cloud network 114 finds aninterface module in its data store 4604, it transmits the interfacemodule to the access device 108 via communication 4616. For example, ifthe access device 108 is currently using a default interface module fornetwork device 102 that was determined based on exchangingcommunications 4610 and 4612, and then subsequently is able to connectto the cloud network 114, communications 4614 and 4616 between theaccess device 108 and the cloud network 114 can be used to obtain anupdated interface module for the network device 102.

Upon receiving communication 4616 from the cloud network 114, the accessdevice 108 populates a record in cache 4602 corresponding to the networkdevice 102 with device type and controls information received viacommunication 4616. That is, when the access device is remote from thelocal area network 100, it can exchange communications 4614 and 4616with the cloud network 114 to receive an interface module for a networkdevice. Information received via communications 4616 can be used topopulate records of cache 4602. Records in cache 4602 can be updatedusing modular interfaces received via communication 4616. In additionalor alternative embodiments, new records can be created in cache 4602when communication 4616 includes a modular interface for a newlydiscovered network device.

Records 4624, 4626, 4628 in cache 4602 store device types, uniqueinterface module IDs, and controls information for known networkdevices. The access device 108 uses the records in cache 4602 to rendervisual interfaces in the display 4622. For example, the display 4622 caninclude a navigable list of modular tiles corresponding to networkdevices in the local area network 100.

Display 4622 can also include an indicator representing a state ofnetwork device 102. In embodiments, communications 4612 and/or 4614 caninclude a last known state of the network device 102 and/or historicaldata associated with the network device 102. In one embodiment, suchstate information can be based on information received via communication4612 from the network device 102 when the access device 108 is connectedto the local area network 100. In this way, display 4622 of the accessdevice 108 can reflect a current state and historical data for thenetwork device 102 when the access device is not connected to the localarea network 100. In additional or alternative embodiments, the stateinformation can be based on information received via communication 4616from the cloud network 114 when the access device 108 is connected tothe cloud network 114. Using the state information, an interface moduleor tile for the network device 102 within display 4622 can indicate an‘on’ or ‘off’ state for the network device 102 when the network deviceis powered on or off.

FIG. 47 illustrates an example of a network 4700, according toembodiments of the present invention. Specifically, the network 4700 canbe a wireless local area network enabling an access device tocommunicate with network devices to control adjustment of attributesrelated to operation of the network devices. Network 4700 includesnetwork device 4702, network device 4704, network device 4706, andnetwork device 4708. The network 4700 also includes access device 108.In other words, the network 4700 may be substantially similar to thenetwork 100 except that access device 108 has been turned on near thenetwork 4700, to which it is associated, or has entered an area to whichthe network 4700 can reach.

When access device 108 can enter the network 4700 as shown in FIG. 47,access device 108 may be authenticated based on the access device'sauthentication with the logical network or may otherwise commencecommunication with cloud network 114. Access device 108 may alsocommunicate notification of its presence or other information directlyto other network devices 4702-4708 within network 4700, as shown in FIG.47 by communication paths 4730. As noted, such communication may includevarious communications protocols, such as Universal Plug and Play(UPnP), Bluetooth®, Zigbee®, Ultra-Wideband (UWB), WiFi-Direct, WiFi,Bluetooth® Low Energy (BLE), sound frequencies, and/or the like. Forexample, access device 108 may communicate to all other devices innetwork 4700, including network device 4702, network device 4704,network device 4706, and network device 4708, information/data regardingits status. Such status data may include the fact that it is present andturned on, or other status data/information. At any time that networkdevices 4702, 4704, 4706 and 4708 recognize that access device 108 ispresent at network 4700, the network devices may communicate back toaccess device 108. For example, the network devices may send anacknowledgement (e.g., ACK signal) back to access device 108 to confirmthat they received the status data sent by access device 108. Thenetwork devices may also send their own status data to access device108.

While network devices 4702-4708 and access device 108 may each receivecommunication from other network devices around the network 4700,including the status of each of those network devices, network devices4702-4708 and/or access device 108 may be continuously scanning network4700 (including, for example, running discovery algorithms) to determinewhether any devices within the network have moved, turned on/off orotherwise added to or subtracted from the network 4700, or haveotherwise changed statuses.

Since network devices 4702-4708 and access device 108 may each receivecommunication from other devices around network 4700, including thestatus of each of those devices, each network device within network 4700may know the status of each other network device in the network 4700.For example, access device 108 or devices 4702-4708 may not be requiredto communicate with cloud network 114 in order to obtain one or more ofsuch statuses. Since cloud network 114 is an external network and may beremote from network 4700, communication between network devices withinthe network 4700 and cloud 114 may take more time than communicationbetween two devices within network 4700. For example, communicationbetween devices within network 4700 may take anywhere from 1 millisecondto 100 milliseconds, while communication between a device within network4700 and the cloud network 114 may take anywhere from 50 milliseconds to1 second or more). Furthermore, if a network device is retrievinginformation from cloud 114, the request must travel from the networkdevice to cloud network 114, and then the information must travel backfrom cloud network 114 to the network device. This process may doublethe latency caused by retrieving information with cloud 114. Therefore,devices within the network 4700 may choose to send and receive/retrievestatuses directly with other devices within the network 4700 instead ofcommunicating such information via cloud network 114. When a networkdevice receives status data from another network device on the device'slocal area network 4700, it may store that status data so that it mayretrieve and use that status data at a later time.

The network 4700 can enable user to monitor and/or control operation ofthe network devices 4702-4708. For example, the user can monitor andcontrol network devices by interacting with a visual interface of thegateway 110 (i.e., a web page for gateway 110) and/or a visual interfacerendered on display 4622 of the access device 108. In some embodiments,an application may be run on the access device 108. The application maycause the access device 108 to present a graphical interface (e.g., thedisplay 4622) that includes a visual interface for each network deviceaccessible on the network 4700.

The visual interface corresponding to a network device can provide a“status” of the network device. In certain embodiments, the status orstate of a network device can be indicated within the tile with textand/or graphically. The status of the network device can change based ontime (e.g., a period, an interval, or other time schedule). The statusof a network device may be any piece of information pertinent to thatparticular network device. The status of a network device may be anychangeable variable of that particular network device. For example, thestatus of a network device may include a state of the network deviceitself (e.g., on or off) or how the network device is situated withinthe network with respect to the other network and other network devicesthroughout the network. For example, the status of a network device mayrefer to the network device's proximity to another network device and/orits ability to communicate with another network device because of therelative signal strength between the two network devices. In certainembodiments, the status can include a value or some other informationindicating a unit of measure for a setting or an attribute related tooperation of a network device. The setting or the attribute can beadjustable within a range of values. For example, the network device canbe a light bulb and the status can include a value corresponding tobrightness (e.g., a percentage of total brightness) emitted by the lightbulb when the light bulb is powered-on. In another example, the networkdevice can be a motion sensor and the status can include a valuecorresponding to sensitivity of the sensor in a range of values between0 to 100 when the sensor is powered-on. In yet another example, thenetwork device can be a fan and the status can include a valuecorresponding to a speed of the fan on a scale of 0 to 100 when the fanis powered-on.

An application operating on the access device 108 can access the cache4602 to obtain information to display the visual interface for eachnetwork device 4702-4708 registered within the network 4700. FIG. 47also illustrates that each of the network devices 4702-4708 may includea storage device, such as a cache, for storing data, including dataregarding its own status and data regarding statuses received from theother network devices within the network 4700.

As explained with reference to FIG. 46, the cache 4602 can contain aknown interface list 4620 including interface information for different,known types of network devices. As explained earlier, the knowninterface list 4620 can include a record for each network device knownby the access device 108 to exist on the network 4700. When anapplication is run on the access device 108, the access device 108 canaccess the known interfaces 4640 in the cache 4602 to present thedisplay 4622. The display 4622 can present one or more visualinterfaces, each corresponding to a network device known to exist on thenetwork 4700. Each visual interface can be generated based on a visualinterface module corresponding to each network device on the network4700. In an example, the display 4622 can include a visual interface(e.g., a module tile) for each network device in the network 4700 havingan interface in the known interface list 4620.

The cache 4602 can also contain a known network device list 4720 with aDevice A (e.g., network device 4702) last known address 4724, a Device B(e.g., network device 4704) last known address 4726, and a Device C(e.g., network device 4706) last known address 4728. The known devicelist 4720 can list each network device known by the access device 108 toexist on the local area network 4700. Network devices 4702-4706 can bepresent on the known device list 4620 because the access device 108 oranother device capable of updating the cache 4602 of the access device108 has previously discovered network devices 4702-4706.

In addition to address information, the cache 4602 can contain knownstatus information 4604 about each network device in the known devicelist 4720. When the application is run on the access device 108, theaccess device 108 can access the known status information in the cache4602 to present a status display 4622. The access device 108 canpopulate each tile with an indicator representing the respective knownstatus information for each network device in the known device list4720. For example, the status display 4622 can include an indicatorrepresenting an “on” state for Device A 4702 and Device C 4706, and anindicator representing “off” for Device B 4704. The status display 4622can include an indicator of one or more attributes, one or moresettings, or a combination thereof related to operation of each networkdevice in the network 4700. For example, the status display can includea brightness of a bulb (e.g., a brightness of 56 in a range of valuesbetween 0 and 100) of the network device 4702 (e.g., a light bulb), avalue of sensitivity of a sensor (e.g., a value of 34 in a range ofvalues 0-100) for the network device 4704 (e.g., a motion sensor), and avalue of brightness (e.g., 65 percent brightness) for the network device4706 (e.g., a light bulb). Although shown as having a single indicatorfor an attribute or a setting related to operation of a network device,the status display 4622 can present a plurality of indicatorscorresponding to different attributes and/or settings related tooperation of a network device.

In some embodiments, the cache 4602 can include other information abouta network device. The other information can indicate a network device'sfirmware version, last known firmware update status, connectivity tocloud status, registration status (e.g., whether the network device hasa key or not), and other such information. The cache 4602 can includeinformation that could be used for troubleshooting. In embodimentsdescribed below, the access device 108 can access status informationfrom another other device on the network 4700 and can use thatinformation to update its own cache 4602, update the status display4622, and/or pass the information to the cloud network 114 and/or thegateway 110 for trouble shooting and/or storage.

In some embodiments, a device, such as the access device 108, maydetermine a current status of one or more network devices. The devicemay determine a current status of a network device based on statusinformation stored locally in the cache, such as the device list 4720.The device may receive updated status from network devices, the cloudnetwork 114, the gateway 110, or combinations thereof. The updatedstatus may be stored in the cache 4602. The updated status may becompared to a current status know by a network device to determinewhether a change has occurred in the status. In some embodiments, anetwork device (e.g., the network device 4702, the network device 4704,or the network device 4706) may determine its own updated status andwhether a change has occurred in its own current status. The statusinformation determined by the network device and sent to an accessdevice for presentation.

In some embodiments, the device (e.g., the access device 108) maydetermine a type of status corresponding to a change in status. The typeof status may correspond to category of status for a device, such as itsavailability status, operation status, communication status, or anupgrade status (e.g., whether it is receiving an upgrade in firmware,hardware, and/or software). The type of status may be indicated by theupdated status. In some embodiments, a device may determine a type ofstatus based on comparison of an updated status to a current status. Theinformation included in the updated status may be processed to determinea type of status. The type of status of an updated status may becompared to a type of status of a current status to determine whether achange in the current status has occurred.

In some embodiments, a device (e.g., the access device 108) maydetermine whether another device (e.g., the network device 4702, thenetwork device 4704, or the network device 4706) is associated with astatus for a time period (e.g., a threshold time period). The timeperiod may correspond to a threshold time period, during which certainevents affecting a status of a network device may be expected to occur.For example, a firmware upgrade to a device may last for a period oftime, which may be considered the threshold period. A status existinglonger than the threshold time period may be presented to a user.Because some events may be expected to last for a period of time, astatus may not be presented to a user when a threshold time period hasnot been reached. In this manner, status updates that worthy attentionof a user, such as those exceeding a threshold time period, may bepresented to a user rather than all status updates. The threshold timeperiod may correspond to a type of status. In some embodiments, thethreshold time period may be chosen based on the type of statusdetermined for an updated status. The device, such as an access device108, may determine whether a status of the network device does notchange for the threshold time period. The device may make such adetermination so that the device can determine whether to present astatus update in the display 4622. To determine whether the thresholdtime period is being met, a device may compare the updated status with aprevious updated status to determine whether an updated status reflectsa change during the threshold time period. The comparison may beperformed periodically as updated status is received or may be performedat the conclusion of a threshold time period.

A device seeking to check the status of any other device on the networkmay communicate with the cloud network 114, to which all devices on thenetwork are connected either directly or indirectly. Since the cloudnetwork 114 and/or the gateway 110 can store an updated table/list ofthe statuses of each of the devices 4702-4706 within the requestingnetwork's local area network, the cloud network 114 and/or gateway 110may communicate such status data to the network devices 4702-4706 andthe access device 108. For example, if network device 4702 and 4706 wereto each turn on and communicate their statuses to cloud network 114,cloud network 114 may analyze the status of network devices 4702 and4706 and communicate to network devices 4702 and 4706 that they are eachconnected to the same local area network.

Throughout FIGS. 48-53, various displays are shown for managingpresentation of a status of a device in accordance with someembodiments. The displays shown in the FIGS. 48-53 may present one ormore graphical or visual interfaces (e.g., a modular tile). Suchinterfaces may be rendered upon execution of a visual interface moduleby an application executing on a computing device, e.g., the accessdevice 108. The visual interfaces may be shown with one or a combinationof modular tiles, interactive areas, interactive elements, icons, links,command regions, windows, toolbars, submenus, drawers, and buttons thatare used to initiate actions, invoke routines, monitor network devices,configure network devices, control network devices, or invoke otherfunctionality. The visual interfaces may display a status of a device,display a current state of a device, display historical data associatedwith a device, display a last known status of a device, receive input toadjust one or more settings and/or one or more attributes related tooperation of a device, receiving input to adjust one or more secondarysettings of a device, enable a user to cause notifications to be sent,or a combination thereof. For brevity, the differences occurring withinthe figures, as compared to previous or subsequent figures, aredescribed and illustrated below for purposes of illustration.

FIG. 48 shows an example of interfaces presented by a device (e.g., theaccess device 108) according to some embodiments. In particular, FIG. 48shows a display 4805 that presents a graphical interface with modulartiles 4802, 4804, 4806 for interacting with network devices in anetwork. The modular tile 4802, modular tile 4804, and the modular tile4806 may correspond to network device 4702, network device 4704, andnetwork device 4706, respectively. A modular tile may enableinteractions to monitor and/or control operation of a network devicecorresponding to the modular tile. Interaction with a modular tile mayenable a user to cause a status of the network device to be updated suchthat the tile may show a current status of the network device. In someembodiments, a modular tile may indicate a current status of a networkdevice and may be modified to show updates in the status.

A modular tile may include interactive areas or interactive elements toenable interaction with the modular tile. In some embodiments, themodular tile may be an interactive area or an interactive element toreceive input to control operation of a network device corresponding tothe tile. The modular tiles 4802, 4804, 4806 may each include respectiveicons 4880, 4882, 4884, and respective names 4840, 4842, 4844corresponding to a network device. For example, the modular tile 4802corresponds to the network device 4702 (e.g., a light bulb) and includesan interactive element depicted as a power button 4830 for turning thenetwork device 4702 on and off. In another example, the modular tile4804 corresponds to the network device 4704 (e.g., a motion sensor) andincludes an interactive element depicted as a power button 4832 forturning the network device 4704 on and off. In another example, themodular tile 4806 corresponds to the network device 4706 (e.g., a lightbulb) and includes an interactive element depicted as a power button4834 for turning the network device 4706 on and off.

An interactive element shown as a control (e.g., power button) in amodular tile may be displayed based on a state or a status of thenetwork device corresponding to the tile including the control. Anappearance of the interactive element may be altered to be visiblyilluminated, colored, shaded, bolded, animated, modified to a differentshape, or a combination thereof. In some embodiments, the interactiveelement may be modified to a modified interactive element. The modifiedinteractive element may have a modified appearance that is differentfrom an appearance of the interactive element before modification.Although depicted as a corresponding to a single network device, eachmodular tile may correspond to multiple network devices, or a group ofnetwork devices. For purposes of illustration, a modular tile may bedescribed herein as enabling control for operation of a network device.However, in some embodiments, a modular tile may enable control foroperation of multiple network devices, some or all of which may bedifferent.

In some embodiments, the modular tile may be an interactive element thatis modified to reflect a status of a network device corresponding to thetile. The tile may be modified to a modified tile. The modified tile mayhave a modified appearance (e.g., a modified graphical appearance) thatis different from an appearance (e.g., a graphical appearance) of thetile before modification. The appearance may be modified to reflect acurrent status or state of a network device corresponding to the tile.An appearance of the tile may be modified. For example, the tile may bemodified to be visibly illuminated, colored, shaded, bolded, animated,modified to a different shape, or a combination thereof.

In some embodiments, a modular tile may be customizable by developersand/or manufacturers of the corresponding network devices. For, example,name 4844 of the modular tile 4804 is customized to indicate the name ofa specific type of sensor (e.g., “My Motion Sensor”). In additional oralternative embodiments, a modular tile may convey status informationabout a corresponding network device or network devices, including, butnot limited to, a firmware version, a last known firmware update status,status for connectivity to a cloud network, an availability of thenetwork device or network devices on the network, a firmware updatestatus, a hardware update state, a software update status, connectivityerrors or issues that the network device(s) may have encountered,communication issues between the network device(s) and other networkdevices, a power state, registration status (i.e., an indication thatthe network device has a key or does not), a primary mode of the networkdevice (i.e., on, off), a secondary mode of the network device (i.e.,standby, high, low, eco), other modes of operation, a schedule, settings(e.g., speed or sensitivity) for the network device, one or moreattributes (e.g., brightness) related to operation of the networkdevice, rules that may be in effect for the network device or related tooperation of the network device, or a combination thereof.

Each of the modular tiles 4802, 4804, 4806 may be chosen and displayedbased on the type and/or capabilities of a network device (e.g., thenetwork devices 4802, 4804, 4806). The display 4805 may be flexible andmay dynamically display the most appropriate modular tile 4802 based onan environment in which the application executes. For instance, thedisplay 4805 may enables a mechanism in which a user may pre-configure amodular tile to display an appropriate message for a network device.

FIG. 48 shows that display 4805 may include interactive elements (e.g.,selectable icons and links) 4814, 4816, 4818, 4820, 4822, and 4824outside of the modular tiles 4802, 4804, 4806. For example, refresh icon4814 may be selected to refresh information presented in display 4805,such as, for example, status and state information displayed in orindicated by the modular tiles 4802, 4804, 4806. For instance, one ormore of the statuses for each of the modular tiles 4802, 4804, 4806,respectively, may be refreshed on an automatic, periodic basis, inaddition to being manually updated when refresh icon 4814 is interactedwith (e.g., selected).

In some embodiments, the display 4805 may include an edit link 4816.Interaction with the edit link 4816 may enable one or more of themodular tiles 4802, 4804, 4806 to be editable. For example, edit link4816 may be selected to sort or re-order the sequence of the modulartiles 4802, 4804, 4806 displayed in the display 4805. Upon interactionwith the edit link 4816, one or more of the modular tiles 4802, 4804,4806 may be interacted with to be disabled or removed. Devices icon 4818may be selected to cause a graphical interface to be presented with oneor more network devices that are discovered in a network.

In some embodiments, the display 4805 may include a rules icon 4820.Interaction with a rules icon 4820 may enable a graphical interface thatenables customization of display rules pertaining to the networkdevices. The graphical interface may provide one or more interactiveelements to control one or more rules related to control of the networkdevices (e.g., the network devices 4702, 4704, 4706). In someembodiments, interaction with the rules icon 4820 may enable a graphicalinterface to enable a user to configure rules of operation, modes ofoperation, notifications, or combinations thereof, any or all of whichmay be stored locally on the network device presenting the display 4805or other network devices (e.g., the network devices 4702, 4704, 4706).The graphical interface may be presented in the display 4805 or may beshown separately from the display 4805.

In some embodiments, the display 4805 may include a news icon 4822.Interaction with the news icon 4822 may enable a graphical interface tobe display which presents news including information associated with thenetwork devices and/or the applications. For instance, interaction withthe news icon 4822 may cause a graphical interface to be displayed inthe access network device to present information related to the networkdevices controlled via the modular tiles 4802, 4804, 4806 and/orinformation relevant to the application, such as notifications ofavailable tile updates.

In some embodiments, the display 4805 may include a more icon 4824.Interaction with the more icon 4824 may provide one or more additionalgraphical interfaces to enable access to additional features forcontrolling and/or monitoring the network devices in a network. Forexample, an additional graphical interface may be displayed to controlone or more settings and/or attributes of any network devices connectedto a network. In another example, an additional graphical interface maybe displayed to configure secondary options to be initiated by aninterface used to control a network device. In yet another example, anadditional graphical interface may be displayed to present networkdevices that have a status indicating unavailability of the networkdevice. In some embodiments, a modular tile may be removed from displayupon determination of unavailability of the network device correspondingto the tile. An example of an additional graphical interface isdescribed with reference to FIG. 53.

The display 4805 may receive input to adjust one or more settings and/orone or more attributes related to operation of a network devicecorresponding to a tile presented in the display 4805. Thefunctionalities may include secondary functionalities, tertiaryfunctionalities, etc. beyond primary functionalities (e.g., powercontrol). The display 4805 may be interactive or may include interactiveareas configured to receive input to assist the user in controllingfunctionality, settings, and/or attributes of a network deviceidentified in the display 4805. The display 4805 may be interactive ormay include interactive areas to enable initiation of actions andadditional actions. For example, each modular tile 4802-4806 may beinteractive to receive input. The input received via a modular tile maybe used to control operation of a network device corresponding to themodular tile. The input may include one or more contacts by a user. Acontact may include a tap, a touch, a click, a swipe, a circular motion,a selection, a continuous movement, other motions, other userinteractions, user movements, or combinations thereof. In someembodiments, the input may be received as a series of inputs to controlthe network device corresponding to the modular tile. The contact mayoccur in one or more directions on the display 4805 or a modular tile.

The adjustment for controlling operation of the network device 4706 mayinclude adjusting an attribute or an setting (e.g., brightness) of thenetwork device 4706. The adjustment to the attribute can be based on anamount of the input, a length of time when the input is received, adistance of movement corresponding to the input, a direction of theinput, or any other data obtained from the input. The network device canbe adjusted from a last known state of operation or last known state ofan attribute or setting.

In some embodiments, the display 4805 may be modified to a display 4810to reflect a current status of a network device corresponding to one ofthe modular tiles 4802-4806. For example, the display 4805 may bemodified to the display 4810 to show a change in current status of thenetwork device (e.g., the network device 4702) corresponding to themodular tile 4802 and the network device (e.g., the network device 4704)corresponding to the modular tile 4804. In some embodiments, a graphicalappearance of a modular tile or one of its components may be modified toreflect a change in status of the network device corresponding to themodular tile. For example, the graphical appearance of the modular tile4802 may be modified to a grey color or shaded to indicate that thenetwork device (e.g., “Light 1”) is not on the network. In someembodiments, a modular tile or its components may be disabled to preventa user from controlling operation of a network device using the modulartile or a component that permits interaction. A modular tile or any ofits components may be disabled to prevent a user from controllingoperation of a network device when a network device has a certain status(e.g., inactive or unavailable on the network). In another example, thegraphical appearance of the modular tile 4804 is modified to a differentcolor or pattern to indicate that the network device (e.g., “My MotionSensor”) is undergoing a firmware update. In some embodiments, agraphical appearance of a modular tile may be displayed in one of manydifferent patterns and/or colors to reflect a type of status. In thedisplay 4810, a graphical appearance of the modular tile 4804 may bemodified differently than the graphical appearance of the modular tile4802 to illustrate a different in types of status,

In some embodiments, the modular tile or a component of a modular tilemay not be modified to reflect a change in status until a threshold timeperiod is reached. The threshold time period may enable a determinationto be made as to whether the change is temporary or due to a brief event(e.g., a firmware update or intermittent problem) as opposed to aproblem or a status that may last much longer. By making such adetermination, the status of a modular tile can be updated to reflectevents of relevance to a user. Whereas indicating a status changeimmediately upon such determination, a user may be burdened and possiblyprevented from using a network device when such status may change again.Therefore, it may be desirable to indicate a change in the currentstatus of a network device upon determining that an updated status of anetwork device is not temporary, but rather may last longer than thethreshold time period. Such a technique may provide assurances to a userthat a status of a network device has in fact changed. In someembodiments, the threshold time may correspond to a type of status. Forexample, a threshold time for a firmware update to a network device maybe 17 seconds corresponding to amount of time needed to perform anupdate. In another example, the threshold time for an inactive networkdevice may correspond to 1 minute, after which a system may be able toconfirm that a network device has in fact become inactive such that thenetwork device may not receive communication.

FIG. 49 shows another example of interfaces presented by a device (e.g.,the access device 108) according to some embodiments. The display 4905may include elements described with reference to FIG. 48, such aselements included in the display 4805. The display 4905 includes agraphical interface with the modular tiles 4802, 4804, 4806. The modulartile 4802, modular tile 4804, and the modular tile 4806 may correspondto network device 4702, the network device 4702, and the network device4706, respectively. Specifically, FIG. 49 shows techniques for removinga modular tile from a display present by a network device. In someembodiments, a modular tile may be removed when an updated status of anetwork device corresponds to certain types of statuses. For example,when a network device becomes inactive, a user may interact with amodular tile corresponding to the network device to request the modulartile to be removed. Removing a modular tile may prevent a user fromattempting to control a network device, which may or may not beoperational depending on the type of status (e.g., inactive orunavailable). Further, by removing modular tiles, organization ofnetwork devices shown on a display may be improved so as to enable auser to identify information about network devices that are operational(e.g., active or available).

In one example, input may be received via a modular tile, such as themodular tile 4802, in the display 4905. The modular tiles 4802-4806 maybe interactive to receive input. For example, input may be received viainteraction with an area (e.g., an area 4950) of the modular tile 4802.In some embodiments, the area may correspond to the button (e.g., thebutton 4830) in a modular tile. In some embodiments, a modular tile orone of its components may be interactive when a corresponding networkdevice has a certain type of status. In this example, the modular tile4802 may have an appearance of the modular tile 4802 in the display4810. The appearance of the modular tile 4802 in the display 4905 mayindicate a certain type of status of a network device, such asunavailability or an inactive state of the network device. The inputreceived in the area 4950 may include one or more contacts with the area4950 by a user. A contact may include a tap, a touch, a click, a swipe,a motion, a selection, a continuous movement, other motions, other typesof interactions, or combinations thereof. In some embodiments, the inputreceived may correspond to a request to remove the modular tile 4802.

In some embodiments, a display, such as the display 4905, may bemodified to the display 4910 to present an additional graphicalinterface 4980 in response to the input received for the modular tile4802. The graphical interface 4980 may be presented to enable a user toconfirm a request to remove the modular tile 4802. The graphicalinterface 4980 may be presented in the display 4910 or in a differentdisplay on the same network device. The graphical interface 4980 mayprovide additional information about a status of a network devicecorresponding to the modular tile requested for removal. One or moreinteractive elements may be included in the graphical interface 4980.For example, an interactive element 4985 (e.g., “Cancel”) in thegraphical interface 4980 may enable a user to cancel a request to removethe modular tile 4802. In another example, an interactive element 4990(e.g., “Hide”) in the graphical interface 4980 may enable a user toconfirm a request to remove the modular tile 4802. By confirming, themodular tile 4802 may be removed. A graphical interface like thegraphical interface 4980 may be presented for other modular tiles inresponse to input received via those modular tiles.

FIG. 50 shows another example of interfaces presented by a device (e.g.,the access device 108) according to some embodiments. The display 5005may include elements described with reference to FIG. 48, such aselements included in the display 4805. The display 5005 includes agraphical interface with the modular tiles 4802, 4804, 4806. The modulartile 4802, modular tile 4804, and the modular tile 4806 may correspondto network device 4702, the network device 4702, and the network device4706, respectively. Specifically, FIG. 50 shows techniques for removinga modular tile from a display present by a device. In some embodiments,a modular tile may be removed when an updated status of a network devicecorresponds to certain types of statuses. The example in FIG. 50illustrates another technique for enabling a user to request a modulartile to be removed.

In one example, input may be received via a modular tile, such as themodular tile 4802, in the display 5005. The modular tiles 4802-4806 maybe interactive to receive input. For example, input may be received viainteraction with an area (e.g., an area 5050) of the modular tile 4802.In some embodiments, the area may correspond to the button (e.g., thebutton 4830) in a modular tile. In some embodiments, a modular tile orone of its components may be interactive when a corresponding networkdevice has a certain type of status. In this example, the modular tile4802 may have an appearance of the modular tile 4802 in the display4810. The appearance of the modular tile 4802 in the display 5005 mayindicate a certain type of status of a network device, such asunavailability or an inactive state of the network device. The inputreceived in the area 5050 may include one or more contacts with the area5050 by a user. In some embodiments, the input received may correspondto a request to remove the modular tile 4802.

In some embodiments, a display, such as the display 5005, may bemodified to the display 5010 to present one or more interactive elements5055, 5060 in response to the input received for the modular tile 4802.The interactive elements 5055, 5060 may be presented to enable a user toconfirm a request to remove the modular tile 4802. The display 5010 maybe modified to provide additional information about a status of anetwork device corresponding to the modular tile requested for removal.In the example of FIG. 50, the interactive element 5055 (e.g., “Cancel”)may enable a user to cancel a request to remove the modular tile 4802.The interactive element 5060 (e.g., “Hide”) may enable a user to confirma request to remove the modular tile 4802. By confirming, the modulartile 4802 may be removed. Interactive elements such as the interactiveelements 5055, 5060 may be presented for other modular tiles in responseto input received via those modular tiles.

FIG. 51 shows another example of interfaces presented by a device (e.g.,the access device 108) according to some embodiments. The display 5105may include elements described with reference to FIG. 48, such aselements included in the display 4805. The display 5105 includes agraphical interface with the modular tiles 4802, 4804, 4806. The modulartile 4802, modular tile 4804, and the modular tile 4806 may correspondto network device 4702, the network device 4702, and the network device4706, respectively. Specifically, FIG. 51 shows other techniques forremoving a modular tile from a display present by a device. In someembodiments, a modular tile may be removed when an updated status of anetwork device corresponds to certain types of statuses. The example inFIG. 51 illustrates another technique for enabling a user to request amodular tile to be removed.

In one example, input may be received via a modular tile, such as themodular tile 4802, in the display 5005. The modular tiles 4802-4806 maybe interactive to receive input. For example, input may be received viainteraction with an area (e.g., an area 5150) of the modular tile 4802.In some embodiments, a modular tile or one of its components may beinteractive when a corresponding network device has a certain type ofstatus. In this example, the modular tile 4802 may have an appearance ofthe modular tile 4802 in the display 5110. The appearance of the modulartile 4802 in the display 4810 may indicate a certain type of status of anetwork device, such as unavailability or an inactive state of thenetwork device. The input received in the area 5050 may include one ormore contacts with the area 5050 by a user. In some embodiments, theinput received may correspond to a request to remove the modular tile4802. In the example shown in FIG. 51, a user may provide inputbeginning with an interaction at the interactive area 5050 andcontinuing to an interactive area 5155 shown in the display 5110. Theinput may correspond to a swipe movement or a motion across the modulartile 4802. In some embodiments, the display 5110 may be modified toenable a user to perform continuous interaction within a modular tile.

The display 5110 may be presented with a modified modular tile based onthe modular tile 4802. The modular tile 4802 may be modified to presentan interactive element 5160 (e.g., “Hide”) in response to the inputcorresponding to the interaction from the interactive area 5150 to theinteractive area 5155. The interactive element 5160 may be interactiveto receive input. Interaction with the interactive element 5160 maycause the modular tile 4802 in the display 5110 to be removed. Thedisplay 5110 may be modified to the display 5115 in which the modulartile 4802 is removed in response to interaction with the interactiveelement 5160.

FIG. 52 shows another example of interfaces presented by a device (e.g.,the access device 108) according to some embodiments. Specifically, FIG.52 shows other techniques for removing a modular tile from a displaypresent by a device. In some embodiments, a modular tile may be removedwhen an updated status of a network device corresponds to certain typesof statuses. The example in FIG. 52 illustrates another technique forenabling a user to request a modular tile to be removed.

The display 5205 may include elements described with reference to FIG.48, such as elements included in the display 4805. The display 5205includes a graphical interface with the modular tiles 4802, 4804, 4806.The modular tile 4802, modular tile 4804, and the modular tile 4806 maycorrespond to network device 4702, the network device 4702, and thenetwork device 4706, respectively. In this example, the modular tile4802 may have an appearance of the modular tile 4802 and the modulartile 4804 may have an appearance of the modular tile 4804 in the display4810. The appearance of the modular tile 4802 and the modular tile 4804in the display 5205 may indicate a certain type of status of a networkdevice. For example, the modular tile 4802 may indicate unavailabilityof the network device 4702 and the modular tile 4804 may indicate afirmware update of the network device 4704.

In the example shown in FIG. 52, a user may interact with the refreshicon 5214 in the display 5205 to refresh information shown in thedisplay 5205. For instance, one or more of the statuses for each of themodular tiles 4802, 4804, 4806, respectively, may be refreshed when therefresh icon 4814 is interacted with (e.g., selected). To refresh theinformation, the device presenting the display may obtain updated statusinformation about the network devices corresponding to the modular tiles4802-4806. As explained above, an access device (e.g., the access device108) may maintain status information about network devices in a knowndevice list 4720. The access device may access the cache 4602 todetermine updated status information about each device in the knowndevice list 4720.

In some embodiments, receiving input via the refresh icon 5214 may causethe modular tiles 4802-4806 presented in the display 5205 to berefreshed. The display 5105 may be modified to the display 5110 todisplay an updated status of network devices corresponding to themodular tiles 4802-4806. In some embodiments, new modular tiles may bepresented corresponding to network devices that are connected to anetwork and/or that have a certain type of status (e.g., available,active, etc.). In some embodiments, one or more of the existing modulartiles 4802-4806 in the display 5105 may be removed based on a type ofstatus. For example, the modular tile 4802 may be removed and notpresented in the display 5210 because the network device correspondingto the modular tile 4802 is inactive or unavailable. However, themodular tile 4804 may not be removed because the sensor corresponding tothe modular tile 4804 is undergoing a firmware update, which may lastfor a short time. In such instances, it may be desirable for modulartiles certain types of statuses to remain in a display. In someembodiments, rather than removing a modular tile, a modular tile may beplaced in a different position in the display 5210, such as at thebottom of an order of modular tiles.

FIG. 53 shows another example of interfaces presented by a device (e.g.,the access device 108) according to some embodiments. Specifically, FIG.53 illustrates graphical interfaces for managing information aboutdevices that are unavailable. For purposes of illustration, FIG. 53 isdescribed according to an example of devices that are unavailable. Thetechniques described for FIG. 53 may be implemented for modular tilescorresponding to network devices having other types of statuses.

The display 5305 may include elements described with reference to FIG.5, such as elements included in the display 4805. The display 5305includes a graphical interface with the modular tiles 4802, 4804, 4806.The modular tile 4802, modular tile 4804, and the modular tile 4806 maycorrespond to network device 4702, the network device 4702, and thenetwork device 4706, respectively. In this example, the modular tile4802 may have an appearance of the modular tile 4802 and the modulartile 4804 may have an appearance of the modular tile 4804 in the display4810. The appearance of the modular tile 4802 and the modular tile 4804in the display 5305 may indicate a certain type of status of a networkdevice. For example, the modular tile 4802 may indicate unavailabilityof the network device 4702 and the modular tile 4804 may indicate afirmware update of the network device 4704.

The display 5305 may include an interactive element 5318 (e.g.,“Unavailable Devices”). Interaction with the interactive element 5318may cause the display 5310 to be generated and presented at a device.The display 5310 may present one or more modular tiles 4802, 5304corresponding, each corresponding to network devices that areunavailable. In some embodiments, one or more of the modular tiles 4802may have been presented in the display 5305. In some embodiments, themodular tile 5304 may have been previously removed from another display.Although the display 5310 shows network devices that are unavailable,the display 5310 may present modular tiles corresponding to networkdevices having other types of statuses. The modular tiles 4802, 5304 maybe added back to a display such as the display 5305 when a status of thenetwork device corresponding to the modular tile changes fromunavailable to another type of status. In some embodiments, the modulartiles shown in a separate display, e.g., the display 5310, may be addedback to the display 5305. For example, a user may interact with one ormore of the modular tiles 4802 and 5304 to cause these them to be addedback to the display 5305. The tiles may be added even if the networkdevices are unavailable or inactive.

FIG. 54 shows flowchart is illustrated of embodiments of a process 5400for displaying a status associated with network devices according tosome embodiments of the present disclosure. Specifically, the process5400 provides a technique for presenting a status of network devices ona network. For example, the process 5400 may be performed by a computingdevice (e.g., the access device 108) to present a status of networkdevices (e.g., the network devices 4702, 4704, 4706) on a network (e.g.,the local network 4700). A computing network device may implement theprocess 5400 regardless of whether the computing network device iswithin a network to which network devices are connected or remotely incommunication with the network.

Process 5400 is illustrated as a logical flow diagram, the operation ofwhich represents operations that may be implemented in hardware,computer instructions, or a combination thereof. In the context ofcomputer instructions, the operations represent computer-executableinstructions stored on one or more computer-readable storage media that,when executed by one or more processors, perform the recited operations.Generally, computer-executable instructions include routines, programs,applications, objects, components, data structures, and the like thatperform particular functions or implement particular data types. Theorder in which the operations are described is not intended to beconstrued as a limitation, and any number of the described operationsmay be combined in any order and/or in parallel to implement theprocesses.

Additionally, the process 5400 may be performed under the control of oneor more computer systems configured with executable instructions and maybe implemented as code (e.g., executable instructions, one or morecomputer programs, or one or more applications) executing collectivelyon one or more processors, by hardware, or combinations thereof. Asnoted above, the code may be stored on a computer-readable storagemedium, for example, in the form of a computer program comprising aplurality of instructions executable by one or more processors. Thecomputer-readable storage medium may be non-transitory.

Beginning at block 5402, the process 5400 may determine that a networkdevice (e.g., network device 4702, network device 4704 or network device4706) is connected to a network (e.g., the network 4700). Determiningthat a network device is connected to a network may include locating aunique identifier associated with the network device. A uniqueidentifier for the network device may be received from the networkdevice itself or from the network. 5402 may be performed as part ofdiscovery process to identify network devices in the network. Theprocess 5400 may be performed by a computing device that is not locatedwithin the network (i.e., at a remote location outside the network).

At block 5404, the process 5400 may determine a current status of thenetwork device. For example, the access device 108 may determine acurrent status of one or more of the network devices 4702, 4704, 4706.The status may be determined based on information (e.g., the device list4720) stored in memory (e.g., the cache 4602) accessible to a computingdevice. The information may indicate the current status of the networkdevice. In some embodiments, the process 5400 may include determiningthe current status of the network device based on an updated statusreceived directly from the network device (e.g., the network device4702, the network device 4704, or the network device 4706), or otherdevices connected to the network. In some embodiments, the currentstatus may be determined from information received from the cloudnetwork 114. The information may include the status of the networkdevices on a network. In some embodiments, the process 5400 may includedetermining the current status from a gateway (e.g., the gateway 110).The current status may be received directly from the gateway orindirectly from the gateway via a cloud network.

In some embodiments, the current status of a network device may indicateinformation about a status of operation associated with the networkdevice, availability of the network device on a network, otherinformation about a status of the network device, or a combinationthereof. For example, a current status of a network device may indicatewhether the network device is operational, one or moresettings/attributes related to operation of the network device, whetherthe network device has encountered any communication errors, whether thenetwork device is undergoing updates (e.g., a firmware update, asoftware update, or a hardware update), or a combination thereof. Inanother example, a current status of a network device may indicatewhether the network device is connected to a network, whether thenetwork device can receive communications from other network devices onthe network (e.g., the access device 108, the network device 4702, thenetwork device 4704, or the network device 4706), whether the networkdevice can communicate with one or more gateways of the network, or acombination thereof. In another example, the current status of a networkdevice may indicate whether the network device can communicate with acloud network (e.g., the cloud network 114).

At block 5406, the process 5400 may include generating a graphicalinterface (e.g., any one of the displays 4622, 4805, 4905, 5005, 5105,5205, 5305, 5310) that includes an interactive element (e.g., themodular tile 4802, the modular tile 4804, or the modular tile 4806)corresponding to a network device (e.g., the network device 4702, thenetwork device 4704, or the network device 4706). The interactiveelement may correspond to a network device on the network. Theinteractive element may indicate a current status associated with thenetwork device corresponding to the interactive element. For example, amodular tile (e.g., the modular tile 4802, the modular tile 4804, or themodular tile 4806) may indicate a current status of a network devicecorresponding to the tile. The current status may be indicated by agraphical appearance of the tile which may indicate a current status byshading, coloring, animation, highlighting, or a combination thereof.The graphical appearance may be modified from that which is shown for adifferent status of the network device. The graphical appearance mayinclude adjusting a size of the tile. At block 5408, the process mayinclude displaying the graphical interface generated at block 5406.

At block 5410, the process 5400 may include determining a change to thecurrent status. The process 5400 may determine a change to the currentstatus by using updated information about a status of the networkdevice. To determine a change, the updated status information about thenetwork device may be compared to the current status. A differenceidentified based on the comparison may indicate a change in the status.In some embodiments, determining the change may include determining atype of status corresponding to the change. The process 5400 maydetermine a type of status based on the updated status information. Thetype of status may for correspond to availability of the network deviceor operation of the network device.

At block 5412, the process 5400 may include determining an updatedstatus associated with the network device. The updated status associatedwith the network device may be determined based on the updated statusinformation.

At block 5414, the process 5400 may include determining that the networkdevice is associated with an updated status for a threshold time period.In order words, the process 5400 may determine whether an updated statusof the network device has not changed (i.e., remains the same) for athreshold time period. The threshold time period may be based on a typeof status. The threshold time period may correspond to a period duringwhich certain events affecting a status of a network device are expectedto last. Those events may correspond to a type of status. For example, athreshold time period may correspond to a brief time period when afirmware upgrade is being performed. The threshold time period maycorrespond to a length of time during which an event should occur thataffects a status of a network device. The threshold time period may bebrief such that updating a modular tile for a network device is notnecessary. In some embodiments, a modular tile corresponding to anetwork device may not be updated with a status of a network device suchas when events occur within the time period. In this example, a modulartile may not be modified during the threshold time period so as to notburden the user with an update that is temporary.

To determine whether the network device is associated with an updatedstatus for the threshold time period, the process 5400 may determinewhether the updated status of the network device changes over thethreshold time period. For example, the process 5400 may process updatedstatus information received about the network device and compare thatinformation to the updated status determined based on the change. Achange in the updated status during the threshold time indicates thatthe network device does not have the updated status for the entirety ofthe threshold time period.

Further, the updated status at any given time during the period may becompared with a previous updated status to determine whether the updatedstatus corresponds to a change in a status of the network device. Bychecking the updated status periodically, the process 5400 can determinewhether the updated status has changed corresponding to a change in astatus of the network device.

At block 5416, the process 5400 includes modifying the graphicalinterface according to the type of status when the threshold time periodis reached. The graphical interface may be modified when the networkdevice has the updated status for a duration of the threshold timeperiod. For example, a modular tile corresponding to a network devicemay be modified to indicate a firmware update if the firmware updatelasts for the threshold time period. The threshold time period mayprevent an update to a modular tile that may burden a user or that maycorrespond to a change in current status that is expected. By modifyingthe graphical interface upon satisfaction of the threshold time period,the graphical interface may provide informative updates that areindicative of problems, more than routine changes (e.g., firmwareupdates) in status. The graphical interface may be modified according toa type of status so as to enable a user to determine a status of anetwork device based on appearance of the tile. In some embodiments, theinteractive element may be modified to indicate the updated status ofthe network device.

It will be appreciated that process 5400 is illustrative and thatvariations and modifications are possible. Steps described as sequentialmay be executed in parallel, order of steps may be varied, and steps maybe modified, combined, added or omitted. Process 5400 may be performedfor each network device that is determined to be connected to thenetwork. It should be noted that process 5400 might be performedconcurrently for other network devices that are connected to thenetwork. Process 5400 may be performed for any number of network devicesthat can be controlled via a graphical interface displayable by acomputing device. Further, the process 5400 may be performed for severalactions that may be performed in response to continued interaction withan interactive area corresponding to a network device.

FIG. 55 shows an example of an interface presented by a device (e.g.,the access device 108) according to some embodiments. In particular,FIG. 55 shows a display 5505 that presents a graphical interface withmodular tiles 5502-5520 for interacting with network devices in anetwork. The modular tiles 5502-5520 may each correspond to a networkdevice. A modular tile may enable interactions to monitor and/or controloperation of a network device corresponding to the modular tile.Interaction with a modular tile may enable a user to initiate one ormore actions in addition to operating a network device corresponding tothe modular tile. The additional action(s) may offer enhancedfunctionality that may not be available from a typical interface (e.g.,a graphical interface or physical interface) that enables a user tocontrol operation of a network device.

Note that an “access device”, such as access device 5508, may be used asan example device that may perform the actions described herein.However, it should be understood that a different device other than anaccess device may perform the same actions, and are within the scope ofthe present technology.

A modular tile may include interactive areas or interactive elements toenable interaction with the modular tile. The modular tiles 5502-5520may each include respective icons 5580-5598, and respective names5540-5558 corresponding to a network device. For example, the modulartile 5502 corresponds to network device (e.g. a light bulb) and includesan interactive element depicted as a power button 5530 for turning thenetwork device on and off. In another example, the modular tile 5504corresponds to a different network device (e.g., a security sensor) andincludes an interactive element depicted as a power button 5532 forturning the network device on and off. A similar principle applies tomodular tiles 5506-5520, and their corresponding network devices. Aninteractive element shown as a control (e.g., power button) may bedisplayed based on a state of the network device corresponding to thecontrol. An appearance of the interactive element may be altered to bevisibly illuminated, colored, shaded, bolded, animated, modified to adifferent shape, or a combination thereof. Although depicted as acorresponding to a single network device, each modular tile maycorrespond to multiple network devices, or a group of network devices.For purposes of illustration, a modular tile may be described herein asenabling control for operation of a network device. However, in someembodiments, a modular tile may enable control for operation of multiplenetwork devices, some or all of which may be different.

In some embodiments, a modular tile may be customizable by developersand/or manufacturers of the corresponding network devices. For, example,name 5544 of the modular tile 5504 is customized to indicate the name ofa specific type of sensor (e.g., “Security Sensor”). In additional oralternative embodiments, a modular tile may convey status informationabout a corresponding network device or network devices, including, butnot limited to, a firmware version, a last known firmware update status,status for connectivity to a cloud network, registration status (i.e.,an indication that the network device has a key or does not), a primarymode of the network device (i.e., on, off), a secondary mode of thenetwork device (i.e., standby, high, low, eco), other modes ofoperation, a schedule, a scene that is enabled using the network device,settings (e.g., speed or sensitivity) for the network device, one ormore attributes (e.g., brightness) related to operation of the networkdevice, additional actions that have been initiated based on interactionwith a modular tile, rules that may be in effect for the network deviceor related to operation of the network device, or a combination thereof.

Each of the modular tiles may be chosen and displayed based on the typeand/or capabilities of a network device associated with each of themodular tiles. The display 5505 may be flexible and may dynamicallydisplay the most appropriate modular tile based on an environment inwhich the application executes. For instance, the display 5505 mayenable a mechanism in which a user may pre-configure a modular tile todisplay an appropriate message for a network device.

FIG. 55 shows that display 5505 may include interactive elements (e.g.,selectable icons and links) 5514 and 5516 outside of the modular tiles.For example, refresh icon 5514 may be selected to refresh informationpresented in display 5505, such as, for example, status and stateinformation displayed in the modular tiles. For instance, one or more ofthe statuses for each of the modular tiles may be refreshed on anautomatic, periodic basis, in addition to being manually updated whenrefresh icon 5514 is interacted with (e.g., selected).

In some embodiments, the display 5505 may include an edit link 5516.Interaction with the edit link 5516 may enable one or more of themodular tiles to be editable. For example, edit link 5516 may beselected to manually sort or re-order the sequence of the modular tilesdisplayed in the display 5505. Upon interaction with the edit link 5516,one or more of the modular tiles may be interacted with to be disabledor removed.

Each modular tile 5502-5520 may be interactive to receive input. Theinput received via a modular tile may be used to control operation of anetwork device corresponding to the modular tile. The input may includeone or more contacts by a user. A contact may include a tap, a touch, apinch, a drag, a click, a swipe, a circular motion, a selection, acontinuous movement, other motions, other user interactions, usermovements, or combinations thereof. In some embodiments, the input maybe received as a single input or a series of inputs to control thenetwork device corresponding to the modular tile. The contact may occurin one or more directions on the display 5505 or a modular tile.

As noted, each of the modular tiles may be chosen and displayed based onthe type and/or capabilities of a network device associated with each ofthe modular tiles. The display 5505 may be flexible and may dynamicallydisplay the most appropriate modular tile based on an environment inwhich the application executes. For example, an access device may onlyinclude a certain amount of physical space to display a certain portionof display 5505. More specifically, only some of the modular tiles5502-5520 may be displayed at one time on the access device for a userto view, and others of the modular tiles 5502-5520 may be hidden fromview (e.g. not displayed on the access device) due to that lack ofspace. For example, as shown in FIG. 55, modular tiles 5502-5520 arelabeled in two groups: modular tile group 5524 and modular tile group5522. Modular tile group 5522 may, for example, be displayed (andvisible) on an access device display at a particular time. Modular tiles5502-5512, which are a part of modular tile group 5522, are representedin FIG. 55 with solid lines to indicate that the modular tiles are shownto be visible on a display of an access device. Modular tile group 5524may, for example, be hidden (and not visible) on an access devicedisplay at that particular time. Modular tiles 5514-5520, which are apart of modular tile group 5524, are represented in FIG. 55 with dashedlines to indicate that the modular tiles are hidden from the display ofthe access device.

The display 5505 may be flexible and may dynamically display the mostappropriate modular tile based on various factors. Therefore, themodular tile groups 5522 and 5524 may change over time, or in otherwords may include different modular tiles at different times dependingon these factors. These factors (or “parameters”) may be related to theuser of the access device, the location of the access device, the typeof the network device associated with a modular tile, the frequency ofuse of the network device associated with a modular tile, the use orfrequency of use of the network device at a particular day or time, thelocation of the network device associated with a modular tile, amongother factors, or a combination of these factors.

Changes to the order or other aspects of the display of modular tileswithin a display such as display 5505 may be based on rules set by auser or other controller of the access device that includes display 5505or may be based on information gathered by the device over time. Forexample, changes to the order or other aspects of the display of modulartiles within a display such as display 5505 may be based on rules set bya user. For example, a user may interact with the display to enable agraphical interface that enables customization of display rulespertaining to the network devices represented by the tiles. Thegraphical interface may provide one or more interactive elements tocontrol one or more rules related to control of the display of the oneor more network devices. These rules may be set to always apply or maybe set to be based on other factors, such as day or time. For example, arule may be set that at 5:00 PM every weekday, certain tiles are visibleto the user if the user opens the display within that time frame. Inanother example, a rule may be set that tile 5502 is always listed atthe top of the visible display screen that is visible by the user. Suchrules may be specific to a particular tile, to a particular networkdevice associated with a tile, to a group of network devices, to one ormore network devices included in a scene, among other possibilities.

In another example, changes to the order or other aspects of the displayof modular tiles within a display such as display 5505 may be based oninformation gathered by the device over time. For example, the accessdevice may collect information regarding one or more of the followingfactors: (1) the tiles that the user has used most often, either over along period of time or over the recent past; (2) the location of theaccess device; (3) the type of the network device associated with amodular tile; (4) the frequency of use of the network device associatedwith a modular tile (5) the use or frequency of use of the networkdevice at a particular day or time; (5) the location of the networkdevice associated with a modular tile; among others.

The access device may collect information related to the tiles that theuser has used most often, either over a long period of time or over therecent past. The data collected may be ranked based on perceived ordetermined relevance (e.g. ore recent data more relevant than older datacollected. The access device may cause display 5505 to display a certaintile or set of tiles that the user has used more than any other tile(s).For example, if the user drives a car often and opens/closes the garagedoor often, then the user may use tile 5520 (garage switch) more oftenthan the reset such that the display 5505 would display tile 5520 to theuser at all times. In another example, the use of certain tiles may betied to a time of year, day of the week, and/or time of day. Forexample, if a user uses tile 5520 most often between certain hours (e.g.early in the morning when leaving for work, and in the evening whenreturning home from work), the access device may cause display 5505 todisplay tile 5520 (e.g. at the top of the list of tiles shown to theuser) during those times. However, if the user views the tiles during adifferent time of day (e.g. lunch time), the display 5505 may display adifferent tile that the user uses most often during that time of day(e.g. tile 5508 because the user checks on the user's security systemduring lunch every day, as noticed by the access device).

The access device may collect information related to the location of theaccess device. For example, if the access device is located outside thehouse (e.g. because the user brought the access device outside the houseas detected by a global positioning system (GPS) or otherwise), thendisplay 5505 may display tiles 5506 (exterior lighting) and 5520 (garageswitch) because the access device may be able to assume that these tilesare most relevant to the user based on the location of the access device(and presumably the location of the user).

The access device may also collect information related to patterns ofbehavior of the user or any other interactions with the display. Forexample, information related to the frequency of use of the networkdevice associated with a modular tile may be used. For example, if theuser uses a certain network device or set of network devices more oftenthan others (either via interacting with the network device via display5505 or otherwise), then the access device may assume that the user ismore likely to use that network device during any given instance of theuser viewing and using display 5505. For example, if the user is indoorsand is close to both security sensor 1 (tile 5504) and interior lighting1 (tile 5502), but the user uses network device associated with tile5504 (security sensor 1) twice as often as the network device associatedwith tile 5502 (light), then the display 5505 may tile 5504 instead oftile 5502. However, the display may also display both tiles 5504 and5502 if both tiles 5502 and 5504 are more likely to be relevant or usedby the user than the other tiles in display 5505.

When determining which information may be relevant to which tile ornetwork device, the type of network device and other factors may beconsidered. For example, certain parameters may be most relevant to agarage door opener network device but may not be relevant for an indoorbasement light switch. For example, time of day and day of week may berelevant to a garage door opener network device, but may not be asrelevant for the indoor basement light switch. In another example,location of the access device (and, for example, the user holding theaccess device) may be highly relevant for the indoor basement lightswitch because if the user is not in the basement, the tile associatedwith that network device may not be relevant to the user at that time.In an alternative embodiment, certain network devices may be relevantbased on their association, or grouping, with other network devices. Forexample, if an indoor light switch is determined to be relevant becausethe user is near (e.g. within a predetermined amount of space of thatnetwork device) that light switch and/or because the user regularly usesthat light switch at that particular day and time of day, then otherlight switches in that room or on that floor may also be determined tobe relevant because the likelihood of the user using those devices atthat time is greater than if the user were not in that room. Networkdevice grouping, or devices in the same scene, as described herein mayalso contribute to grouping for tile ordering and listing purposes.

The access device may also use other actions taken by the user tocollect information relevant to which tiles the display 5505 should showthe user. For example, the access device may use information inputted bythe user, such as bibliographic information related to the user'spersonal choices, names, pictures, or other information inputted relatedthe network devices connected to the network and associated with eachtile, or other information. In addition, the access device may useinformation gathered by other parts of the access device other than theuser's interaction with the display 5505. For example, as noted,information collected from use of the GPS within the access device todetermine the location of the access device at any given time. Inanother example, information related to the user's personal preferencesgathered from the user's use of other applications on the access devicemay be used. As noted, rules may also be created by the user to instructthe access device (or display 5505) to act in a certain way (e.g.display certain tiles before others, possibly only in certain situationsor at certain times).

Since display 5505 may be dynamic, and may adjust based on theinformation collected about the user and use of the access device,display 5505 may display a message to the user to indicate that a changeis being made (e.g. a rule is being changed) based on other informationit collected, or may ask the user if the user would like to change acertain rule or setting based on such information. In such a situation,the user may elect to either make the suggested change or to not makethe suggested change.

Based on all of the information collected and used to determine whichtiles should be visible to the user at a certain time, the tiles (and,in turn, the network devices associated with the tiles) may be given aranking (or “rating” or “score”) regarding its importance to the user atthat time. FIG. 56 shows an example of two interfaces presented by adevice (e.g., the access device 5508) according to some embodiments. Inparticular, FIG. 56 shows a display 5603 that presents a graphicalinterface with modular tiles for interacting with network devices in anetwork and rankings 5601 associated with each tile, according toembodiments of the present technology. Rankings 5601 may indicate one ormore different aspects of the relevance or usefulness of the networkdevices associated with each modular tile. For example, the rankings5601 may be based on any information collected by the access device asdescribed herein. For example, the rankings may be based on a determinedlikelihood of use of each tile or network device associated with eachtile. For example, the tile representing the motion sensor may beassigned a ranking of “1” based on one or more of the factors and typesof information described herein, such as a user's frequent use of themotion sensor compared to other tiles, the location of the accessdevice, the use or frequency of use of the network device at theparticular day or time that the display 5603 is displaying the tiles,the location of the motion sensor, among other factors. The ranking maybe based on these factors as compared to the same or similar factorsrelevant to other tiles or network devices associated with those tiles,such as security sensor 2. Security sensor 2 may be assigned a rankingof “2”, as shown in FIG. 56 in display 5603 based on factors orparameters that indicated that security sensor 2 may not be as relevantto the user as the motion sensor, but more relevant than the other tilesin display 5603 or the network devices associated with those tiles.However, both the motion sensor and security sensor 2 may be shown tothe user in display 5603 because the display can show the user more thanone tile at a time. However, interior lighting 4, assigned a ranking of“10”, may not be visible to the user since that tile is indicated as theleast relevant to the user at that time, and display 5603 may not beable to show the motion sensor tile, security sensor 2 tile, andinterior lighting 4 tile all at the same time due to a lack of space onthe access device display. However, for certain devices, the display ofthe device may be large enough to display all ten tiles. On the otherhand, a user may own more than ten network devices, and therefore mayinclude more than ten modular tiles on display 5603, again requiringthat certain tiles not be shown to the user at a certain time.

Tiles may also be moved within display 5603 such that the tiles arelisted in a different order. For example, the tiles may be listed inorder of ranking as assigned by the rankings 5601, or the tiles may belisted in a different order based on different factors (e.g. groups,scenes, etc.). However, it may not be preferable to change the listingorder of the tiles while a user is viewing display 5603 because suchchanges on the fly may confuse the user. As an alternative, tiles (orgroups of tiles, such as group 5522) may be set to only change uponstart-up of the application that displays display 5605, or for exampleafter the user has left the display (or just not viewed the display) fora certain predetermined period of time.

FIG. 56 also shows a display 5605 that presents a graphical interfacewith modular tiles 5602-5620 for interacting with network devices in anetwork. The modular tiles 5602-5620 may each correspond to a networkdevice. Display 5605 has been adjusted from, for example, display 5505,to show a different set of tiles to the user. In other words, group 5522of visible tiles has shifted to include modular tiles 5606-5616.Similarly, the modular tiles that are not visible to a user have alsoshifted. For example, group 5524 has shifted to include only tiles 5618and 5620. A new group of modular tiles 5626 not visible to the userincludes tiles 5602 and 5604. Groups 5522 and 5524 may have shifted (andgroup 5626 created) because of changes in how the different tiles andtheir corresponding network devices have changed in ranking, for examplewithin rankings 5601. Rankings 5601 may have shifted in a variety ofdifferent ways. For example, as a general matter, rankings 5601 may haveshifted because the information collected and used to determine rankings5601 may have changed. For example, rankings 5601 may have shiftedbecause the user moved to a different location (e.g. within theirhouse), the display 5605 may be viewed at a different time than display5505 was being viewed, a network device may have moved location, one ormore new network devices may have been added to the network, one or morenetwork devices may have been removed from the network, among otherpossible changes. In other words, if any information collected by theaccess device changes over time, the rankings 5601 may change, which maycause the display to shift and a different set of tiles to be visible tothe user. More specifically, due to the change in rankings, the tilesmay shift positions on the display or order in the list of tiles. Thisallows the list to stay in whatever prior order but increases theavailability of the likeliest devices to be selected. As an alternativeor in addition, the list may shift down or up within the display so thata different set of tiles are visible to the user. This allows the listto stay in whatever prior order but increases the availability of thelikeliest devices to be selected by the user.

FIG. 57 shows a flow chart including an example method of displayingdynamic rankings of modular tiles and their associated network devices,according to embodiments of the present technology. The example methodbegins with steps 5702 and 5704. Step 5702 includes receiving originaldata associated with one or more network devices on a network. Step 5704includes generating additional original data associated with one or morenetwork devices on a network. The data in both steps 5702 and 5704 mayalso be associated with tiles associated with the network devices. Thedata may be associated with a history of a user's interaction with thenetwork device(s) or a user's interaction with one or more tilesassociated with the network device(s). The process may also functionwith only step 5702 or step 5704.

Step 5706 may include assigning original rankings ot each of the one ormore network devices based on the received and/or generated data. Theserankings may dynamically change over time, which may change the orderthe network devices (or their associated tiles) are listed, as describedfurther with respect to steps 5708-5718.

Step 5708 may include receiving or generating (or both) new dataassociated with the one or more network devices on the network. This newdata may include new data or data that has changed from the last timedata corresponding to the network devices (or tiles associated with thenetwork devices) were received or generated. Step 5710 includesgenerating new rankings based on the original data, additional data, andnew data. In other words, step 5710 includes updating the rankings basedon the new data. Step 5712 includes comparing the new rankings to theoriginal rankings, and step 5714 includes determining whether the newrankings are different than the original rankings. Steps 5712 and 5714may be completed at the same time or may be the same step.

In step 5716, if the new rankings are different than the originalrankings, tiles associated with the network devices may be displayed inorder of the new rankings. The new and original rankings may bedetermined to be “different” if they are different in any respect, or ifthey are different enough such that they cross a predetermined thresholdof difference. In step 5718, if the new rankings are determined to notbe different than the original rankings, tiles associated with thenetwork devices may be displayed in order of the original rankings.However, step 5718 may be completed such that the result is to displaythe tiles in order of the new rankings (e.g. if the new rankings are thesame as or only slightly different than the original rankings, but thenew rankings more accurately reflect the current state or relevance ofthe network devices and their associated modular tiles.

FIGS. 58-62 relate to techniques for controlling operation of networkdevices, in accordance with some embodiments. The techniques may includemanaging information about settings for operation of network devices.The techniques may include controlling the interfaces presented at adevice (e.g., an access device) to control operation of network devices.The techniques may include implementation of one or more datastructures. Each data structure may be implemented using one or moretypes of data structures, including without limitation, an array, alinked list, a hash table, a tree, or other programming structure thatstores data.

In FIG. 58, data structure is depicted for storing information relatedto controlling operation of network devices. The information storedusing the data structure may be stored locally on an access device, forexample, the information stored in onboard storage (e.g., local cache)of the access device. The information may include device-specificinformation, such as information about the network devices to which anaccess device is connected, status information about the networkdevices, group information about the network devices, and/or interfaceinformation for displaying interface(s) to control operation of thenetwork devices. The information stored in the data structure(s) mayenable an access device to dynamically present interfaces forcontrolling operation of many different types of network devices, someof which may have different settings and/or attributes for theiroperation. The device information stored on an access device may bereceived from network devices, a network including the network devices,and/or a remote network that has access to a network including thenetwork devices. The device information may be received as part of aregistration process and updates about the device.

FIG. 58 depicts a data structure 5810 that is stored on a device, suchas an access device. Data structure 5810 may store device informationabout network devices connected to a network. For example, datastructure 5810 may include a field 5820 (“Device Identifier”) that hasdevice identifier information for network devices accessible to anaccess device, such as network device A (“Device A”), network device B(“Device B”), network device C (“Device C”), and network device D(“Device D”). One or more of the network devices identified by thedevice identifiers may be different.

Data structure 5810 may include a field 5830 that has device typeinformation (e.g., a value indicating a device type) indicating a devicetype for each of the devices identified in field 5820. The value may bea numeric value that is defined for a device type. For example, field5830 may have a value indicating “a light bulb” device type for DeviceA, a value indicating “a heater” device type for Device B, a valueindicating “a heater” device type of Device C, and a value indicating “asensor” device type of Device D.

Data structure 5810 may store interface information for different devicetypes of known network devices. Data structure 5810 may include a field5840 that has interface information indicating an interface identifier(ID) (e.g., a unique interface module ID). The interface ID may beassociated with one or more interfaces (e.g., a graphical interface)that are defined for a device type. The interface ID may be used toretrieve a modular interface for displaying at an access device for anetwork device. In some embodiments, the access device can access anetwork (e.g., a remote network) to obtain interfaces for networkdevices. In the example shown in FIG. 58, field 5840 may include aninterface identifier (“Interface_ID1”) for Device A, an interfaceidentifier (“Interface_ID2”) for Device B, an interface identifier(“Interface_ID3”) for Device C, and an interface identifier(“Interface_ID4”) for Device D. Based on the devices for whichinterfaces are to be presented at an access device, the interfaceidentifier can be used to retrieve an interface to display for thedevices.

Data structure 5810 may include a field 5850 that has controlinformation about the devices identified by device identifier 5820. Forexample, the control information in field 5850 for a device may includesettings information about one or more settings for operation of thedevice. In some embodiments, the settings information may include areference (e.g., a location in memory) to a different data structurethat stores settings information for a device. Examples of settingsinformation are described with reference to FIGS. 60 and 61. Thesetting(s) may correspond to capabilities of a device. The settingsinformation may include a value corresponding to the setting. Thesettings may include primary functionality (e.g., a primary setting), asecondary functionality (e.g., a secondary setting), a tertiaryfunctionality (e.g., a tertiary setting), and any other dependentfunctionality.

The interfaces for a device type may be based on the capabilities of thedevice. In some embodiments, multiple interfaces may be defined fordifferent devices that have the same device type. The different devicesmay have different capabilities although being associated with the samedevice type. The capabilities of the devices may be determined byinformation provided by a manufacturer of the devices. In someembodiments, a device may be modified with logic (e.g., software,hardware, firmware, or a combination thereof) to access hardware (e.g.,a host board) on the device to determine capabilities of the device.Determining the capabilities may include determining values for one ormore settings and/or attributes of the device. Based on thecapabilities, interfaces may be defined to control operation of adevice. The interfaces may include interactive elements that enableadjustment of the settings. The scale of the settings may or may notcorrespond to the values defined by the capabilities corresponding tothose settings. As such, an access device or the device may performprocessing to translate or convert values of settings defined for thedevice.

In some embodiments, some of the devices for which information is storedin data structure 5810 may be associated with a group of devices. Agroup of devices may have devices that have different device types. Assuch the interfaces presented for different devices in a group may bethe individual interfaces displayed for those devices and/or interfacesthat provide interactive elements to control operation of some or all ofthe devices in a group. When devices have different capabilities, andthus, different settings, an access device may perform processing todetermine common capabilities or to determine operation of differentcapabilities of the devices as a group.

To control operation of devices, one or more data structures may becreated to store information including settings information aboutcapabilities of those devices. The data structures may bedevice-specific and may be stored according to a setting hierarchicaldata structure, such as hierarchical data structure 5900 in FIG. 59. Inat least one embodiment, a hierarchical data structure 5900 may bedevice-specific such that it is defined for a device type. As will bedescribed further below, data structure 5900 may be created based on atemplate of settings for a device type. As discussed above, settings maybe predefined based on a type of device. The settings for a device typemay be provided by a network device for which the settings apply, or maybe determined by another device (e.g., an access device). The value ofeach setting may be stored using a data structure that has settingsinformation for the setting.

Data structure 5900 may be defined for different network devices thathave the same device type. Data structure 5900 may be used to determinewhether a conflict exists in controlling operation of a group of networkdevices, some of which have different setting hierarchy although havingthe same device type.

Data structure 5900 may be created differently for different devicetypes. The examples shown in FIG. 60, depict details of the data storedfor settings within data structure 5900. Data structure 5900 may becreated for each network device that has a device type. Each instance ofdata structure 5900 may vary based on the capabilities of the device forwhich data structure 5900 is created, such that it may include fewer ormore of the data structures in data structure 5900 depending on thecapabilities of the device. Data structure 5900 can be defined accordingto alternative arrangements such that data structure 5900 is notarranged as a hierarchical structure, but has an arrangement thatillustrates the relationships between the data structures shown in FIG.59.

Data structure 5900 may have a data structure 5950 (“root node”) thatstores device-specific information for a device having a device typedefined by data structure 5900. Root node 5950 may store informationabout a device including its device type and other information about thecapabilities (e.g., a device model). The information in root node 5950may be used to determine the data structures in data structure 5900.

Root node 5950 may include a reference (e.g., a location in memory) toone or more data structures 5902, 5904, each for a different primarysetting for a device having the device type indicated by root node 5950.For example, data structure 5900 may have multiple data structures 5902,5904, each for a different primary setting (e.g., 1^(st) primarysetting, . . . nth primary setting). In the example of FIG. 59, datastructure 5902 represents a primary setting for a network device. Theprimary setting may correspond to a primary capability of the networkdevice, one that may not depend on another setting for operation of thenetwork device. A capability may be a primary capability as othersettings (e.g., secondary settings or tertiary settings) may bedependent on adjustment of the primary capability. For example, aheater's primary capability may be power, such that its power setting(e.g., power on or power off) control power operation of the heater.

A data structure for a primary setting of a network device may indicatea reference to one or more data structures for one or more secondarysettings of a network device. A secondary setting for a network devicemay be a setting that controls operation of the network device, but thatis dependent on a primary setting having a particular value. Forexample, a heater may have a primary setting for power, and a secondarysetting for heating modes (e.g., low, medium, high, and frost protect).In the example of FIG. 59, data structure 5902 indicates a reference totwo data structures, data structure 5912 and data structure 5914, eachfor a different secondary setting, and data structure 5916 (e.g., asecondary setting for an n^(th) primary setting) may have a secondarysetting 5916. Each data structure for a secondary setting may include avalue for the secondary setting of a secondary capability of a networkdevice.

A data structure for a secondary setting of a network device mayindicate a reference to one or more data structure for one or moretertiary settings of a network device. A tertiary setting for a networkdevice may be a setting that controls operation of the network device,but that is dependent on a secondary setting having a particular value.For example, a heater may have a tertiary setting for temperature thatis activated for certain values of a secondary setting for heatingmodes. The range of values that can be adjusted for the temperature maybe limited based on the different heating modes. In the example of FIG.59, secondary setting data structure 5912 has a reference to twotertiary settings corresponding to data structure 5922 and datastructure 5924. Data structure 5914 has a reference to a single tertiarysetting corresponding to data structure 5926.

Although not shown, data structure 5900 may have any number of levels,each corresponding to a different level of settings. For purposes ofillustration, data structure 5900 is shown having an n^(th) level ofsetting correspond to data structure 5932, which is referenced by datastructure 5922. The n^(th) level of setting may correspond to a settingthat is dependent on a n^(th)−1 level of setting for a network device.

Now turning to FIG. 60, data structures are shown for managing operationof network devices. In the example shown in FIG. 60, data structure 5950is created for a device type of a heater. Data structure 5950 may storeinformation about a type of device including other device informationsuch as model and manufacturer information about the device. In someinstances, network devices may have the same device type, but may bedifferent by model, manufacturer, or other criteria such that theirsetting relationship is not identical. The settings hierarchy may bedefined based on the device information in data structure 5950. Thesettings hierarchy may be used to standardize the settings of networkdevices, including those that have similar or identical capabilities.

Data structure 5950 may include a reference to a primary capabilitydefined by a primary setting (e.g., a power setting). In the example ofFIG. 60, data structure 5950 includes a reference to data structure 5902for a primary setting for the heater. Data structure 5950 may storeinformation about a primary setting, among others, such as a capabilityidentifier (“Power”), a range (“0-1”), a current value of the setting(“1”), or a reference to a secondary setting if one is enables based onthe primary setting. The value of a setting according to a range of thesetting may correspond to a state of the setting for a network device.

The capability identifier may be used to reference the setting for whicha value applies. When an access device is adjusting a network device,the access device may transmit the capability identifier with the valueof the setting. In some embodiments, other information about a settingmay be transmitted with the value of the setting.

The network device may determine operation of a network device based onthe data transmitted about a setting. The setting hierarchy can be usedto determine whether a value of a setting is permitted based on acurrent value of the setting or other settings from which a setting isdependent on. In some embodiments, a data structure storing settinginformation may include information indicating whether a conflict existswith the setting or another other setting.

In the example of FIG. 60, data structure 5902 includes a reference todata structure 5912 for a secondary setting. In this example, datastructure 5912 represents setting information for a secondary settingcorresponding to a mode of operation that is enabled when the powersetting for the heater is in a power on state. Data structure 5912 mayinclude a capability identifier (“Mode”), a range (e.g., 0—low,1—medium, 2—high, 3—frost-protect), and a current value of the setting(“1”). Data structure 5912 may include a dependent capability thatindicates the values on which a tertiary setting depends.

Data structure 5922 in the example of FIG. 60 may represent settinginformation for a tertiary setting corresponding to a temperaturecapability of a heater when specific modes (e.g., low, medium, or high)are enabled. Data structure 5922 may include a capability identifier(“Temperature”), a range (e.g., 0-100 degrees), and a current value ofthe setting (e.g., “50”). Data structure 5922 may not include adependent capability therefore there may not be a reference to anotherdata structure. A tertiary setting for a heater may be enabled when themode is any one of the values 0-2. Data structure 5912 includes areference to data structure 5922 for the tertiary setting.

FIGS. 61 and 62 depict processes for controlling operations of networkdevices, in accordance with some embodiments. These processes can beimplemented by a device, such as an access device that can controloperation of network devices.

In FIG. 61, a process 6100 is depicted for adjusting a setting of anetwork device using one or more data structures (e.g., a settinghierarchy). Process 6100 may begin at step 6102 where device informationis determined for a network device. Device information may be obtainedfrom local storage of an access device that implements process 6100,from a local network that includes the network device, a remote network,or the network device itself. Device information may indicateinformation about the network device such as a device type and settingsinformation.

At step 6104, a current value of a setting of a network device isdetermined. An access device may determine the current value of one ormore settings for a network device. The current value may be determinedusing the device information, such as information stored in datastructure 5810. A record in data structure 5810 corresponding to anetwork device may be obtained. The control information 5850 for arecord may be used to reference settings information in one or more datastructures, such as any of the data structures in FIG. 60. The datastructures for settings can be traversed to identify a current value foreach setting, or a specific setting.

At step 6106, an interface is displayed to control operation of one ormore network devices. The interface may be selected based on a devicetype of each network device. In some embodiments, an interface ID for anetwork device may be used to retrieve one or more interfaces thatenable control of the network device. The settings informationdetermined at step 6104 may be used to determine a state of operationbased on each setting of a network device. In some instances, a settingmay have a zero value or some other value indicating that the setting isnot active. The interface may be selected for display based on thesettings that are active. The interface may display settings informationfor a particular level of settings based on traversing the datastructures storing settings information.

At 6108, an input may be received to adjust a setting of a networkdevice displayed at step 6106. The input may be received because of oneor more interactions with the interface. The setting may be active orinactive on the interface. The interface may be displayed with multiplesettings, including a setting in a lowest level in the hierarchy ofsettings for a network device that has multiple levels of settings.Based on interaction with the interface displayed for a network device,input may be received to change a value of a setting.

At step 6110, the setting for which input is received at step 6108 maybe adjusted based on the input. The input may correspond to a valuebased on adjustment of values for a setting in the interface. The valuesfor a setting displayed in the interface may correspond to a range ofvalues for the setting defined for the network device. In someembodiments, the scale of the values for the setting (e.g., a range ofvalues between 0 and 10) may be different from the range of the valuesdefined for a setting. As such, the input may correspond to a value of asetting to be adjusted to a range of values. The value indicated by theinput may be adjusted (e.g., converted) to a range of values defined forthe setting.

Adjusting a setting based on the input may include transmitting data tocause operation of the network device to be adjusted based on thesetting. The data may include the value of the setting to adjust or thevalue after the adjustment. The value may be transmitted with anidentifier of the setting (e.g., a capability identifier) so that thenetwork device knows the setting to adjust. The information aboutadjusting the setting may be transmitted directly to the network deviceor through another device or network (e.g., a local network or a remotenetwork).

At step 6112, a related setting may be identified based on adjusting thesetting at step 6110. The data structures storing setting informationfor a network device can be used to determine whether one or moreadditional settings are activated. For example, a primary setting thatis adjusted to a particular value may cause one or more secondarysettings (e.g., a related setting) to be active. Based on adjusting thesetting at step 6110, a related setting (e.g., a secondary setting ortertiary setting) can be identified. A related setting may be identifiedbased on the value of the setting adjusted.

At step 6114, an interface displayed at step 6106 may be updated todisplay a related setting to control operation of the network device.Using an interface identifier of the network device, one or moreinterfaces for the related setting can be retrieved. The interface at anaccess device may be updated to display an interface corresponding tothe related setting. For example, an interface corresponding to therelated setting may be presented in a submenu (e.g., peekaboo drawer).The related setting may be presented and enabled when the value of thesetting is adjusted to a value at which the related setting can beenabled. In some embodiments, the interface may be displayed with all orpart of the interface displayed at 6106. The interface may be displayedwith a current value of the related setting obtained from the settinginformation for the related setting.

The interface may be displayed with one or more interactive elementscorresponding to other settings that are related to the related setting.The setting(s) related to the related setting may be obtained fromsetting information in the data structures storing setting information.The value of the related setting can be analyzed to determine whetherthe value satisfies a threshold for enabling or disabling the othersettings that are related.

Process 6100 may end after step 6114.

In FIG. 62, a process 6200 is depicted for adjusting a setting of agroup of network devices using one or more data structures (e.g., asetting hierarchy). Process 6200 may begin at step 6202, by determiningdevice information for a group of network devices. As disclosed herein,network devices may be organized as a group. Device information gatheredfor network devices in a group may be stored to indicate a relationship.For example, data structures described with reference to FIGS. 58-60 fordifferent network device may be stored in association with each other toindicate a relationship as a group of network devices.

At step 6204, a current value of a setting of each of the networkdevices in a group may be determined. At step 6206, a group setting maybe determined for a group of network devices. As disclosed herein, agroup of network devices may include similar or different networkdevices. The setting hierarchy may be used to determine a setting thatcan be designated as a group setting. In some embodiments, a groupsetting may be a setting that is common or similar to each of thenetwork devices in the group. A common or similar setting betweennetwork devices may be one which corresponds to the same capability foreach of those network devices. Using the setting hierarchy (e.g., datastructure 5900) for each different type of device, processing can beperformed to determine a setting that is common, such as a powersetting, which each network device may have. A device, such as an accessdevice, can search through the data structures of a setting hierarchyfor each network device to compare the settings until a common/similarsetting is identified. Such a determination can be predetermined suchthat data is stored indicating compatibility between network devices,such as common/similar settings. In some embodiments, an access devicecan determine that no group setting exists for a group of networkdevices. Based on such a determination, the interface may be updated toindicate that the group of devices do not have a common/similar setting.The setting hierarchy can be updated with information to indicate thesetting of each network device that belongs to the group.

At step 6208, an interface may be displayed to control operation of agroup of network devices. The interface ID for each network device maybe used to determine the interface for each of the network devices.Interfaces may be identified based on a group setting for a group ofnetwork devices. A common/similar setting, once identified, can bepresented in an interface enabling control of that setting for theentire group of network devices. Upon controlling the group setting,different interfaces can be displayed for each of the different networkdevices in the group upon adjustment of the group setting.

At step 6210, one or more inputs may be received to adjust the groupsetting, displayed at step 6208. The input may be received because ofone or more interactions with the interface. The input may be anadjustment for the group setting displayed in the interface. Forexample, where a group setting is a power setting, a user may interactwith the interface to adjust the power setting to a power on position.At step 6212, the group setting may be adjusted based on the input.Continuing with the previous example, the data structure for a powersetting for each of the network devices in the group may be adjusted tothe value corresponding to the adjustment with the power settingdisplayed in the interface. In some instances where network devices aredifferent device types, the value for the common/similar setting may bebased on a different scale or range. Therefore, the value correspondingto the scale of the group setting may be converted to a scale of thesetting of each network device and each network device in the group maybe adjusted according to the converted value for the setting withrespect to the scale of the network device. The value to adjust thesetting may be stored in a data structure that stores settinginformation and may be sent to the network devices in the group foradjusting their operation.

At step 6214, the interface to control operation of the group of devicesmay be updated. Based on adjusting the group setting for the networkdevices, an interface may be retrieved for the network devices that areadjusted for the group setting. An interface may be retrieved for eachof the network devices, where the interface is one which displaysinteractive to controls to operate each of the network devices based onadjustment of the group setting. An interface for a network device mayenable control of a setting that is enabled based on adjustment of thegroup setting. For example, where the group setting is a power setting,each interface of the network devices may display a control to adjustthe setting that is enabled based on adjusting the power setting. One ormore of the interfaces for the network devices may be displayed in theinterface displayed at step 6208, or may be displayed individually. Insome embodiments, a single interface may be displayed with interactiveelements to control operation of each of the network devices in thegroup.

Process 6200 may end after step 6214.

Substantial variations may be made in accordance with specificrequirements. For example, customized hardware might also be used,and/or particular elements might be implemented in hardware, software(including portable software, such as applets, etc.), or both. Further,connection to other access or computing devices such as networkinput/output devices may be employed.

Substantial variations may be made in accordance with specificrequirements. For example, particular elements might be implemented inhardware, software (including portable software, such as applets, etc.),or both. Further, connection to other access or computing devices suchas network input/output devices may be employed.

In the foregoing specification, aspects of the invention are describedwith reference to specific embodiments thereof, but those skilled in theart will recognize that the invention is not limited thereto. Variousfeatures and aspects of the above-described invention may be usedindividually or jointly. Further, embodiments can be utilized in anynumber of environments and applications beyond those described hereinwithout departing from the broader spirit and scope of thespecification. The specification and drawings are, accordingly, to beregarded as illustrative rather than restrictive.

In the foregoing description, for the purposes of illustration, methodswere described in a particular order. It should be appreciated that inalternate embodiments, the methods may be performed in a different orderthan that described. It should also be appreciated that the methodsdescribed above may be performed by hardware components or may beembodied in sequences of machine-executable instructions, which may beused to cause a machine, such as a special-purpose processor or logiccircuits programmed with the instructions to perform the methods. Thesemachine-executable instructions may be stored on one or more machinereadable mediums, such as CD-ROMs or other type of optical disks, floppydiskettes, ROMs, RAMs, EPROMs, EEPROMs, magnetic or optical cards, flashmemory, or other types of machine-readable mediums suitable for storingelectronic instructions. Alternatively, the methods may be performed bya combination of hardware and software.

Where components are described as being configured to perform certainoperations, such configuration can be accomplished, for example, bydesigning electronic circuits or other hardware to perform theoperation, by programming programmable electronic circuits (e.g.,microprocessors, or other suitable electronic circuits) to perform theoperation, or any combination thereof.

While illustrative embodiments of the application have been described indetail herein, it is to be understood that the inventive concepts may beotherwise variously embodied and employed, and that the appended claimsare intended to be construed to include such variations, except aslimited by the prior art.

What is claimed is:
 1. A gateway device, comprising: one or moreprocessors; a non-transitory machine-readable storage medium containinginstructions which when executed on the one or more data processors,cause the one or more processors to perform operations including:detecting a network device connected to the gateway device, wherein thegateway device provides communication routing for the network device;and probing the network device for meta-information and controlcapabilities of the network device; and a receiver configured to receivethe meta-information and the control capabilities of the network device,wherein the meta-information includes identifying information of thenetwork device; wherein the non-transitory machine-readable storagemedium further contains instructions which when executed on the one ormore data processors, cause the one or more processors to performoperations including: determining a device specific interface object forthe network device using the meta-information; determining one or moreprimary control capabilities corresponding to the network device; andtransmitting a device specific interface object module and the one ormore primary control capabilities, wherein the device specific interfaceobject module and the one or more primary control capabilitiesfacilitate display of a gateway device interface, wherein the gatewaydevice interface displays the device specific interface object and acontrol object, wherein the control object allows control of the networkdevice using the one or more primary control capabilities, and whereinthe one or more primary control capabilities include less than allcontrol capabilities of the network device that are provided in aproprietary interface of the network device.
 2. The gateway device ofclaim 1, wherein the meta-information includes the device specificinterface object module.
 3. The gateway device of claim 1, wherein themeta-information includes a unique identifier for the network device,and further comprising instructions which when executed on the one ormore data processors, cause the one or more processors to performoperations including: transmitting the unique identifier to a cloudserver; receiving information identifying a type of the network device;and determining the device specific interface object for the networkdevice using the information identifying the type of the network device.4. The gateway device of claim 1, wherein determining the devicespecific interface object for the network device further comprises:monitoring behavior characteristics of the network device; determining atype of the network device based on the monitored behaviorcharacteristics; and determining the device specific interface objectfor the network device using the determined type of the network device.5. The gateway device of claim 1, further comprising instructions whichwhen executed on the one or more data processors, cause the one or moreprocessors to perform operations including: transmitting themeta-information of the network device to a second network device. 6.The gateway device of claim 1, wherein the device specific interfaceobject includes the control object, and wherein the device specificinterface object is selectable to control the network device using theone or more primary control capabilities.
 7. The gateway device of claim1, wherein the gateway device interface includes a proprietary interfaceobject, wherein the proprietary interface object is selectable to launchthe proprietary interface of the network device to allow execution of acontrol capability outside of the primary control capabilities.
 8. Thegateway device of claim 1, wherein the gateway device interface displaysinformation corresponding to a plurality of network devices connected tothe gateway device.
 9. A computer-implemented method, comprising:detecting, by a gateway device, a network device connected to thegateway device, wherein the gateway device provides communicationrouting for the network device; probing the network device formeta-information and control capabilities of the network device;receiving the meta-information and the control capabilities of thenetwork device, wherein the meta-information includes identifyinginformation of the network device; determining a device specificinterface object for the network device using the meta-information;determining one or more primary control capabilities corresponding tothe network device; and transmitting a device specific interface objectmodule and the one or more primary control capabilities, wherein thedevice specific interface object module and the one or more primarycontrol capabilities facilitate display of a gateway device interface,wherein the gateway device interface displays the device specificinterface object and a control object, wherein the control object allowscontrol of the network device using the one or more primary controlcapabilities, and wherein the one or more primary control capabilitiesinclude less than all control capabilities of the network device thatare provided in a proprietary interface of the network device.
 10. Themethod of claim 9, wherein the meta-information includes the devicespecific interface object module.
 11. The method of claim 9, wherein themeta-information includes a unique identifier for the network device,and further comprising: transmitting the unique identifier to a cloudserver; receiving information identifying a type of the network device;and determining the device specific interface object for the networkdevice using the information identifying the type of the network device.12. The method of claim 9, wherein determining the device specificinterface object for the network device further comprises: monitoringbehavior characteristics of the network device; determining a type ofthe network device based on the monitored behavior characteristics; anddetermining the device specific interface object for the network deviceusing the determined type of the network device.
 13. The method of claim9, further comprising: transmitting the meta-information of the networkdevice to a second network device.
 14. The method of claim 9, whereinthe device specific interface object includes the control object, andwherein the device specific interface object is selectable to controlthe network device using the one or more primary control capabilities.15. The method of claim 9, wherein the gateway device interface includesa proprietary interface object, wherein the proprietary interface objectis selectable to launch the proprietary interface of the network deviceto allow execution of a control capability outside of the primarycontrol capabilities.
 16. The method of claim 9, wherein the gatewaydevice interface displays information corresponding to a plurality ofnetwork devices connected to the gateway device.
 17. A computer-programproduct tangibly embodied in a non-transitory machine-readable storagemedium of a gateway device, including instructions configured to causeone or more data processors to: detect a network device connected to thegateway device, wherein the gateway device provides communicationrouting for the network device; probe the network device formeta-information and control capabilities of the network device; receivethe meta-information and the control capabilities of the network device,wherein the meta-information includes identifying information of thenetwork device; determine a device specific interface object for thenetwork device using the meta-information; determine one or more primarycontrol capabilities corresponding to the network device; and transmit adevice specific interface object module and the one or more primarycontrol capabilities, wherein the device specific interface objectmodule and the one or more primary control capabilities facilitatedisplay of a gateway device interface, wherein the gateway deviceinterface displays the device specific interface object and a controlobject, wherein the control object allows control of the network deviceusing the one or more primary control capabilities, and wherein the oneor more primary control capabilities include less than all controlcapabilities of the network device that are provided in a proprietaryinterface of the network device.
 18. The computer-program product ofclaim 17, wherein the meta-information includes the device specificinterface object module.
 19. The computer-program product of claim 17,wherein the meta-information includes a unique identifier for thenetwork device, and further comprising instructions configured to causethe one or more data processors to: transmit the unique identifier to acloud server; receive information identifying a type of the networkdevice; and determine the device specific interface object for thenetwork device using the information identifying the type of the networkdevice.
 20. The computer-program product of claim 17, whereindetermining the device specific interface object for the network devicefurther comprises: monitoring behavior characteristics of the networkdevice; determining a type of the network device based on the monitoredbehavior characteristics; and determining the device specific interfaceobject for the network device using the determined type of the networkdevice.